Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 1480768 Details for
Bug 1625223
tempest test tempest.api.volume.admin.test_volume_types.VolumeTypesTest.test_volume_crud_with_volume_type_and_extra_specs is failing for IBM block stroage plugin for iSCSI protocol
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
volume log
volume.log (text/plain), 4.59 MB, created by
atiwary
on 2018-09-04 12:27:58 UTC
(
hide
)
Description:
volume log
Filename:
MIME Type:
Creator:
atiwary
Created:
2018-09-04 12:27:58 UTC
Size:
4.59 MB
patch
obsolete
>2018-08-22 02:10:18.999 39124 INFO cinder.volume.flows.manager.create_volume [req-84688215-e368-4f22-a800-398fc5506a22 2ed0ad3336214896bb24e0acd5ac41c6 9b40ad2799a24e0b8b34b0df39232752 - default default] Volume b6c86641-4d0d-4245-aa34-de8282595aa0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b6c86641-4d0d-4245-aa34-de8282595aa0'} >2018-08-22 02:10:19.834 39124 INFO cinder.volume.flows.manager.create_volume [req-84688215-e368-4f22-a800-398fc5506a22 2ed0ad3336214896bb24e0acd5ac41c6 9b40ad2799a24e0b8b34b0df39232752 - default default] Volume volume-b6c86641-4d0d-4245-aa34-de8282595aa0 (b6c86641-4d0d-4245-aa34-de8282595aa0): created successfully >2018-08-22 02:10:19.837 39124 INFO cinder.volume.manager [req-84688215-e368-4f22-a800-398fc5506a22 2ed0ad3336214896bb24e0acd5ac41c6 9b40ad2799a24e0b8b34b0df39232752 - default default] Created volume successfully. >2018-08-22 02:10:23.088 39124 INFO cinder.volume.manager [req-6a2b2ad6-26e9-411d-8dce-78c5d1a80dbd 2ed0ad3336214896bb24e0acd5ac41c6 9b40ad2799a24e0b8b34b0df39232752 - default default] Deleted volume successfully. >2018-08-22 02:10:34.259 39124 INFO cinder.volume.flows.manager.create_volume [req-e14f50a9-6ce4-4067-a6f3-25488be25df5 dd2890deeede4d8bac2665e57b6a6eab 00f69a181dad42eeb224b0e8cb956b10 - default default] Volume 24832e8b-59e4-41ae-bb80-ef3788aeb0e9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-24832e8b-59e4-41ae-bb80-ef3788aeb0e9'} >2018-08-22 02:10:34.806 39124 INFO cinder.volume.flows.manager.create_volume [req-e14f50a9-6ce4-4067-a6f3-25488be25df5 dd2890deeede4d8bac2665e57b6a6eab 00f69a181dad42eeb224b0e8cb956b10 - default default] Volume volume-24832e8b-59e4-41ae-bb80-ef3788aeb0e9 (24832e8b-59e4-41ae-bb80-ef3788aeb0e9): created successfully >2018-08-22 02:10:34.810 39124 INFO cinder.volume.manager [req-e14f50a9-6ce4-4067-a6f3-25488be25df5 dd2890deeede4d8bac2665e57b6a6eab 00f69a181dad42eeb224b0e8cb956b10 - default default] Created volume successfully. >2018-08-22 02:10:38.160 39124 WARNING cinder.quota [req-7182f7bf-a03b-49f6-8668-40be89e69dc5 dd2890deeede4d8bac2665e57b6a6eab 00f69a181dad42eeb224b0e8cb956b10 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1719892387 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1719892387, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 02:10:38.161 39124 WARNING cinder.quota [req-7182f7bf-a03b-49f6-8668-40be89e69dc5 dd2890deeede4d8bac2665e57b6a6eab 00f69a181dad42eeb224b0e8cb956b10 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1719892387 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1719892387, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 02:10:38.414 39124 INFO cinder.volume.manager [req-7182f7bf-a03b-49f6-8668-40be89e69dc5 dd2890deeede4d8bac2665e57b6a6eab 00f69a181dad42eeb224b0e8cb956b10 - default default] Deleted volume successfully. >2018-08-22 02:10:48.326 39124 INFO cinder.volume.flows.manager.create_volume [req-62659c7e-d240-4bc0-9a15-1d96cad76dcd 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] Volume c981e055-80c8-48b7-b5ec-50070a99d6cf: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c981e055-80c8-48b7-b5ec-50070a99d6cf'} >2018-08-22 02:10:48.865 39124 INFO cinder.volume.flows.manager.create_volume [req-62659c7e-d240-4bc0-9a15-1d96cad76dcd 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] Volume volume-c981e055-80c8-48b7-b5ec-50070a99d6cf (c981e055-80c8-48b7-b5ec-50070a99d6cf): created successfully >2018-08-22 02:10:48.868 39124 INFO cinder.volume.manager [req-62659c7e-d240-4bc0-9a15-1d96cad76dcd 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] Created volume successfully. >2018-08-22 02:10:50.697 39124 INFO cinder.volume.manager [req-64cc0bad-b217-4646-814c-e45dfb96a04d 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] Retype volume completed successfully. >2018-08-22 02:10:54.628 39124 WARNING cinder.quota [req-c4dfc64e-bbcb-450b-bd68-2a9d6c57bb66 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-996481200 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-996481200, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 02:10:54.629 39124 WARNING cinder.quota [req-c4dfc64e-bbcb-450b-bd68-2a9d6c57bb66 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-996481200 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-996481200, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 02:10:54.883 39124 INFO cinder.volume.manager [req-c4dfc64e-bbcb-450b-bd68-2a9d6c57bb66 8c82d87f7ab14642829f447404d625de a37ce2b73e65474f91ba1eabdaceb0cb - default default] Deleted volume successfully. >2018-08-22 02:11:35.807 39124 INFO cinder.volume.flows.manager.create_volume [req-50f21a8d-e3e5-438f-80f4-866139379dfa 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume 369254f3-745c-49e8-be6b-af13211348e4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-369254f3-745c-49e8-be6b-af13211348e4'} >2018-08-22 02:11:36.336 39124 INFO cinder.volume.flows.manager.create_volume [req-50f21a8d-e3e5-438f-80f4-866139379dfa 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume volume-369254f3-745c-49e8-be6b-af13211348e4 (369254f3-745c-49e8-be6b-af13211348e4): created successfully >2018-08-22 02:11:36.340 39124 INFO cinder.volume.manager [req-50f21a8d-e3e5-438f-80f4-866139379dfa 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Created volume successfully. >2018-08-22 02:11:37.488 39124 INFO cinder.volume.manager [req-a2a100cb-7e8d-439b-8c48-56b77fce18cd 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Attaching volume 369254f3-745c-49e8-be6b-af13211348e4 to instance 1874ad4d-ec8e-409c-8dfa-63a38b5be2ef at mountpoint /dev/vdb on host None. >2018-08-22 02:11:37.561 39124 INFO cinder.volume.manager [req-a2a100cb-7e8d-439b-8c48-56b77fce18cd 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Attach volume completed successfully. >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager [req-22648e16-335c-4091-a30d-9a6e2a89436e e4c824a78d2e49bdb3eb750df50c139f fe18157af8814b7ab94473ccd9943edc - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager force=force) >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-22 02:11:38.811 39124 ERROR cinder.volume.manager >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server [req-22648e16-335c-4091-a30d-9a6e2a89436e e4c824a78d2e49bdb3eb750df50c139f fe18157af8814b7ab94473ccd9943edc - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 02:11:38.813 39124 ERROR oslo_messaging.rpc.server >2018-08-22 02:11:38.944 39124 INFO cinder.volume.manager [req-c88929d0-4b9b-4a66-9c8b-76c78c1ee23e 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Detaching volume 369254f3-745c-49e8-be6b-af13211348e4 from instance 1874ad4d-ec8e-409c-8dfa-63a38b5be2ef. >2018-08-22 02:11:39.044 39124 INFO cinder.volume.manager [req-c88929d0-4b9b-4a66-9c8b-76c78c1ee23e 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Detach volume completed successfully. >2018-08-22 02:11:40.926 39116 INFO cinder.volume.flows.manager.create_volume [req-d36c05b1-9217-4636-abd0-b549a2b593c2 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume 05b02b41-29c5-44dc-9cd8-e4bc1ebec459: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-05b02b41-29c5-44dc-9cd8-e4bc1ebec459'} >2018-08-22 02:11:41.458 39116 INFO cinder.volume.flows.manager.create_volume [req-d36c05b1-9217-4636-abd0-b549a2b593c2 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume volume-05b02b41-29c5-44dc-9cd8-e4bc1ebec459 (05b02b41-29c5-44dc-9cd8-e4bc1ebec459): created successfully >2018-08-22 02:11:41.462 39116 INFO cinder.volume.manager [req-d36c05b1-9217-4636-abd0-b549a2b593c2 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Created volume successfully. >2018-08-22 02:11:44.212 39116 INFO cinder.volume.manager [req-e6f8a515-46bd-4917-abe0-5044b475c1e8 e4c824a78d2e49bdb3eb750df50c139f fe18157af8814b7ab94473ccd9943edc - default default] Deleted volume successfully. >2018-08-22 02:11:44.780 39124 INFO cinder.volume.flows.manager.create_volume [req-376bd8a2-d7a9-4ca7-8400-d83469611f48 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume 033a8062-cd94-4191-9754-ebe36fe9ce83: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-033a8062-cd94-4191-9754-ebe36fe9ce83'} >2018-08-22 02:11:45.317 39124 INFO cinder.volume.flows.manager.create_volume [req-376bd8a2-d7a9-4ca7-8400-d83469611f48 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume volume-033a8062-cd94-4191-9754-ebe36fe9ce83 (033a8062-cd94-4191-9754-ebe36fe9ce83): created successfully >2018-08-22 02:11:45.321 39124 INFO cinder.volume.manager [req-376bd8a2-d7a9-4ca7-8400-d83469611f48 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Created volume successfully. >2018-08-22 02:11:47.481 39124 INFO cinder.volume.manager [req-fd2c83d3-bbc5-4729-ab9c-9f6c9e9ef85f e4c824a78d2e49bdb3eb750df50c139f fe18157af8814b7ab94473ccd9943edc - default default] Deleted volume successfully. >2018-08-22 02:11:47.844 39124 INFO cinder.volume.flows.manager.create_volume [req-21498ccb-1023-4f9b-bdcf-a04f0a18d437 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume 04bd4ca3-3ad0-4c1d-8569-cbecb70df5f0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-04bd4ca3-3ad0-4c1d-8569-cbecb70df5f0'} >2018-08-22 02:11:48.371 39124 INFO cinder.volume.flows.manager.create_volume [req-21498ccb-1023-4f9b-bdcf-a04f0a18d437 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume volume-04bd4ca3-3ad0-4c1d-8569-cbecb70df5f0 (04bd4ca3-3ad0-4c1d-8569-cbecb70df5f0): created successfully >2018-08-22 02:11:48.374 39124 INFO cinder.volume.manager [req-21498ccb-1023-4f9b-bdcf-a04f0a18d437 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Created volume successfully. >2018-08-22 02:11:50.552 39124 INFO cinder.volume.manager [req-c6e34684-8dc2-4c5b-99b0-375a6580ef28 e4c824a78d2e49bdb3eb750df50c139f fe18157af8814b7ab94473ccd9943edc - default default] Deleted volume successfully. >2018-08-22 02:11:51.024 39124 INFO cinder.volume.flows.manager.create_volume [req-be2a4614-6b91-4b16-9037-5c9e0dee20ff 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume 6f0c1ac8-58ef-4614-b7fc-37a370970194: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6f0c1ac8-58ef-4614-b7fc-37a370970194'} >2018-08-22 02:11:51.551 39124 INFO cinder.volume.flows.manager.create_volume [req-be2a4614-6b91-4b16-9037-5c9e0dee20ff 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume volume-6f0c1ac8-58ef-4614-b7fc-37a370970194 (6f0c1ac8-58ef-4614-b7fc-37a370970194): created successfully >2018-08-22 02:11:51.555 39124 INFO cinder.volume.manager [req-be2a4614-6b91-4b16-9037-5c9e0dee20ff 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Created volume successfully. >2018-08-22 02:11:53.860 39124 INFO cinder.volume.manager [req-49ff5268-1e79-45d4-ba76-196788c936ee e4c824a78d2e49bdb3eb750df50c139f fe18157af8814b7ab94473ccd9943edc - default default] Deleted volume successfully. >2018-08-22 02:11:53.980 39124 INFO cinder.volume.flows.manager.create_volume [req-a910ece4-926a-42b2-9643-e170607d4ad7 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume f65c88d0-2573-481e-b741-5eb520d8b359: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f65c88d0-2573-481e-b741-5eb520d8b359'} >2018-08-22 02:11:54.509 39124 INFO cinder.volume.flows.manager.create_volume [req-a910ece4-926a-42b2-9643-e170607d4ad7 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Volume volume-f65c88d0-2573-481e-b741-5eb520d8b359 (f65c88d0-2573-481e-b741-5eb520d8b359): created successfully >2018-08-22 02:11:54.513 39124 INFO cinder.volume.manager [req-a910ece4-926a-42b2-9643-e170607d4ad7 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Created volume successfully. >2018-08-22 02:11:57.262 39124 INFO cinder.volume.manager [req-033d9a44-cfbb-4f4c-8d59-b0836f61af34 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Deleted volume successfully. >2018-08-22 02:11:58.076 39124 INFO cinder.volume.manager [req-14fdb53c-312c-40e1-b48a-4570cf594e2a 2d56622c98274f0da83716f76fa778f0 4e2762b7b4ef4ccd91946c000dfee6ae - default default] Deleted volume successfully. >2018-08-22 02:12:04.497 39124 INFO cinder.volume.flows.manager.create_volume [req-6ce3174d-d2eb-4885-9c9f-b2d4f87b7d48 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Volume fd5e51dc-3f11-4cb5-a2b4-e3b3ca070e9e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fd5e51dc-3f11-4cb5-a2b4-e3b3ca070e9e'} >2018-08-22 02:12:05.031 39124 INFO cinder.volume.flows.manager.create_volume [req-6ce3174d-d2eb-4885-9c9f-b2d4f87b7d48 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Volume volume-fd5e51dc-3f11-4cb5-a2b4-e3b3ca070e9e (fd5e51dc-3f11-4cb5-a2b4-e3b3ca070e9e): created successfully >2018-08-22 02:12:05.035 39124 INFO cinder.volume.manager [req-6ce3174d-d2eb-4885-9c9f-b2d4f87b7d48 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Created volume successfully. >2018-08-22 02:12:06.695 39124 INFO cinder.volume.flows.manager.create_volume [req-cfcac2a3-b544-4449-905d-9a4767c5c372 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Volume 77b41ce3-58f4-447b-beee-374d1c8733c4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-77b41ce3-58f4-447b-beee-374d1c8733c4'} >2018-08-22 02:12:07.223 39124 INFO cinder.volume.flows.manager.create_volume [req-cfcac2a3-b544-4449-905d-9a4767c5c372 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Volume volume-77b41ce3-58f4-447b-beee-374d1c8733c4 (77b41ce3-58f4-447b-beee-374d1c8733c4): created successfully >2018-08-22 02:12:07.226 39124 INFO cinder.volume.manager [req-cfcac2a3-b544-4449-905d-9a4767c5c372 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Created volume successfully. >2018-08-22 02:12:08.740 39116 INFO cinder.volume.flows.manager.create_volume [req-48c6c8b1-7591-4213-bb14-e3e619577fd2 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Volume d5e4e6fc-c412-4c88-917f-08e352207f81: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d5e4e6fc-c412-4c88-917f-08e352207f81'} >2018-08-22 02:12:09.304 39116 INFO cinder.volume.flows.manager.create_volume [req-48c6c8b1-7591-4213-bb14-e3e619577fd2 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Volume volume-d5e4e6fc-c412-4c88-917f-08e352207f81 (d5e4e6fc-c412-4c88-917f-08e352207f81): created successfully >2018-08-22 02:12:09.308 39116 INFO cinder.volume.manager [req-48c6c8b1-7591-4213-bb14-e3e619577fd2 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Created volume successfully. >2018-08-22 02:12:10.344 39124 INFO cinder.volume.flows.manager.create_volume [req-a19f91be-8666-4a07-9a2f-164576b69196 21079a17b3a84d3098eabac1cd3d03ab 3a5a2e8e4ce241e1847756bf43ab86ec - default default] Volume 573ee227-1ea0-45de-a240-e3c93df948a0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-573ee227-1ea0-45de-a240-e3c93df948a0'} >2018-08-22 02:12:11.045 39124 INFO cinder.volume.flows.manager.create_volume [req-a19f91be-8666-4a07-9a2f-164576b69196 21079a17b3a84d3098eabac1cd3d03ab 3a5a2e8e4ce241e1847756bf43ab86ec - default default] Volume volume-573ee227-1ea0-45de-a240-e3c93df948a0 (573ee227-1ea0-45de-a240-e3c93df948a0): created successfully >2018-08-22 02:12:11.048 39124 INFO cinder.volume.manager [req-a19f91be-8666-4a07-9a2f-164576b69196 21079a17b3a84d3098eabac1cd3d03ab 3a5a2e8e4ce241e1847756bf43ab86ec - default default] Created volume successfully. >2018-08-22 02:12:15.303 39124 INFO cinder.volume.manager [req-2e804aa4-d561-47d1-813b-5c775c8818ce 21079a17b3a84d3098eabac1cd3d03ab 3a5a2e8e4ce241e1847756bf43ab86ec - default default] Deleted volume successfully. >2018-08-22 02:12:15.323 39124 INFO cinder.volume.manager [req-3f5071e3-06bf-493d-8e23-347822abf224 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Deleted volume successfully. >2018-08-22 02:12:15.467 39124 INFO cinder.volume.manager [req-1858cf48-01b4-4f33-a3bf-bfafa1d4ce7d 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Deleted volume successfully. >2018-08-22 02:12:15.848 39116 INFO cinder.volume.manager [req-950f536d-32f8-4a45-a690-64782ee9dd08 3cdb760837014c2383a8ab40270824af 67c8dbc0e2ab4fb590969d2cbf5abbd6 - default default] Deleted volume successfully. >2018-08-22 02:12:21.505 39124 INFO cinder.volume.flows.manager.create_volume [req-f90570cf-78e4-467c-9367-8a67f171f4ad 0fcfce832b88400c971fa00cb9fd89de a38d900cea434678bb3c30a5cdebae27 - default default] Volume 9520a4ae-0d13-434c-8fc1-7cc11471e6f2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9520a4ae-0d13-434c-8fc1-7cc11471e6f2'} >2018-08-22 02:12:22.199 39124 INFO cinder.volume.flows.manager.create_volume [req-f90570cf-78e4-467c-9367-8a67f171f4ad 0fcfce832b88400c971fa00cb9fd89de a38d900cea434678bb3c30a5cdebae27 - default default] Volume volume-9520a4ae-0d13-434c-8fc1-7cc11471e6f2 (9520a4ae-0d13-434c-8fc1-7cc11471e6f2): created successfully >2018-08-22 02:12:22.202 39124 INFO cinder.volume.manager [req-f90570cf-78e4-467c-9367-8a67f171f4ad 0fcfce832b88400c971fa00cb9fd89de a38d900cea434678bb3c30a5cdebae27 - default default] Created volume successfully. >2018-08-22 02:12:26.560 39124 INFO cinder.volume.manager [req-0cbec9fa-e564-46d0-a226-bb2a68e41d49 0fcfce832b88400c971fa00cb9fd89de a38d900cea434678bb3c30a5cdebae27 - default default] Deleted volume successfully. >2018-08-22 02:12:34.063 39124 INFO cinder.volume.flows.manager.create_volume [req-8756ab4d-62b6-43c9-80ea-667426aba352 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] Volume 31b68de2-1d28-4640-8d2c-b9658ff65e23: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-31b68de2-1d28-4640-8d2c-b9658ff65e23'} >2018-08-22 02:12:34.609 39124 INFO cinder.volume.flows.manager.create_volume [req-8756ab4d-62b6-43c9-80ea-667426aba352 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] Volume volume-31b68de2-1d28-4640-8d2c-b9658ff65e23 (31b68de2-1d28-4640-8d2c-b9658ff65e23): created successfully >2018-08-22 02:12:34.614 39124 INFO cinder.volume.manager [req-8756ab4d-62b6-43c9-80ea-667426aba352 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] Created volume successfully. >2018-08-22 02:12:36.984 39124 INFO cinder.volume.manager [req-01ef83a9-920a-43fa-8376-350eb7afb57e 88bef64389d64e89a90c7375c0d1a29b dd4ede0e7ed4442387bf0c37a741bf48 - default default] Transfer volume completed successfully. >2018-08-22 02:12:39.369 39124 INFO cinder.volume.manager [req-500847c4-2531-4f0b-a6a9-7683b127ea34 c72e530ee7224e57964d98777c463252 4988f9e2687e49efad19c08a3c04b521 - default default] Deleted volume successfully. >2018-08-22 02:12:40.217 39124 INFO cinder.volume.flows.manager.create_volume [req-aebb89a9-17ea-4d04-bfa4-180f1317b5ab 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] Volume 102efedc-e026-496d-b2d5-0ab4a20975e4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-102efedc-e026-496d-b2d5-0ab4a20975e4'} >2018-08-22 02:12:40.761 39124 INFO cinder.volume.flows.manager.create_volume [req-aebb89a9-17ea-4d04-bfa4-180f1317b5ab 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] Volume volume-102efedc-e026-496d-b2d5-0ab4a20975e4 (102efedc-e026-496d-b2d5-0ab4a20975e4): created successfully >2018-08-22 02:12:40.765 39124 INFO cinder.volume.manager [req-aebb89a9-17ea-4d04-bfa4-180f1317b5ab 490d95d38108489981b6a12456899a5a 9aa2695c8c674c9b97c7a8c3dc6c08ba - default default] Created volume successfully. >2018-08-22 02:12:42.995 39124 INFO cinder.volume.manager [req-023d6771-d6b1-490e-8aa6-04b5bf3035ae c72e530ee7224e57964d98777c463252 4988f9e2687e49efad19c08a3c04b521 - default default] Deleted volume successfully. >2018-08-22 02:12:50.447 39124 INFO cinder.volume.flows.manager.create_volume [req-b48f37d0-1f0f-4bb5-ad2e-566b708c908d 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Volume 84468b82-2d4b-4b53-ae2e-9a4bc9c80e88: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-84468b82-2d4b-4b53-ae2e-9a4bc9c80e88'} >2018-08-22 02:12:51.230 39124 INFO cinder.volume.flows.manager.create_volume [req-b48f37d0-1f0f-4bb5-ad2e-566b708c908d 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Volume volume-84468b82-2d4b-4b53-ae2e-9a4bc9c80e88 (84468b82-2d4b-4b53-ae2e-9a4bc9c80e88): created successfully >2018-08-22 02:12:51.234 39124 INFO cinder.volume.manager [req-b48f37d0-1f0f-4bb5-ad2e-566b708c908d 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Created volume successfully. >2018-08-22 02:12:58.168 39124 INFO cinder.volume.manager [req-c414e8d7-35f6-42f1-9b56-6b939a0ff301 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Attaching volume 84468b82-2d4b-4b53-ae2e-9a4bc9c80e88 to instance ad92dfdd-6789-4947-b43a-2253abfb815a at mountpoint /dev/vdb on host None. >2018-08-22 02:12:58.240 39124 INFO cinder.volume.manager [req-c414e8d7-35f6-42f1-9b56-6b939a0ff301 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Attach volume completed successfully. >2018-08-22 02:12:58.658 39124 INFO cinder.volume.manager [req-461813f9-0942-4dba-9abe-274f2f6f9d3c 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Detaching volume 84468b82-2d4b-4b53-ae2e-9a4bc9c80e88 from instance ad92dfdd-6789-4947-b43a-2253abfb815a. >2018-08-22 02:12:58.750 39124 INFO cinder.volume.manager [req-461813f9-0942-4dba-9abe-274f2f6f9d3c 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Detach volume completed successfully. >2018-08-22 02:13:05.260 39124 INFO cinder.volume.manager [req-b3aff279-cf29-435b-9418-f44e899235db 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Attaching volume 84468b82-2d4b-4b53-ae2e-9a4bc9c80e88 to instance bb5b1abc-8702-47ae-87ad-12e3e02cad8a at mountpoint /dev/vdb on host None. >2018-08-22 02:13:05.448 39124 INFO cinder.volume.manager [req-b3aff279-cf29-435b-9418-f44e899235db 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Attach volume completed successfully. >2018-08-22 02:13:05.942 39124 INFO cinder.volume.manager [req-524c8eac-0232-4ce9-b98d-6ca1e82f0ba0 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Detaching volume 84468b82-2d4b-4b53-ae2e-9a4bc9c80e88 from instance bb5b1abc-8702-47ae-87ad-12e3e02cad8a. >2018-08-22 02:13:06.026 39124 INFO cinder.volume.manager [req-524c8eac-0232-4ce9-b98d-6ca1e82f0ba0 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Detach volume completed successfully. >2018-08-22 02:13:20.017 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-fc1c1721-8b72-45b6-a684-41137454c11e 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] initialize_connection: Did not find a preferred node for volume volume-84468b82-2d4b-4b53-ae2e-9a4bc9c80e88. >2018-08-22 02:13:20.729 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 02:13:38.985 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-fc1c1721-8b72-45b6-a684-41137454c11e 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:13:39.494 39124 INFO cinder.volume.manager [req-fc1c1721-8b72-45b6-a684-41137454c11e 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Copy volume to image completed successfully. >2018-08-22 02:13:43.398 39124 INFO cinder.volume.manager [req-01e8dc07-34a7-45cf-8a90-fab459df3961 289e674a39f04caf880fa0d8a1febac3 377f40a5899c41bd9204afebafca32e0 - default default] Deleted volume successfully. >2018-08-22 02:13:50.201 39124 INFO cinder.volume.flows.manager.create_volume [req-ac80f37f-4588-40c1-8d1d-3c5609c02e46 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Volume d63ab5c0-3b4d-4b47-abe4-15273a454841: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d63ab5c0-3b4d-4b47-abe4-15273a454841'} >2018-08-22 02:13:51.066 39124 INFO cinder.volume.flows.manager.create_volume [req-ac80f37f-4588-40c1-8d1d-3c5609c02e46 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Volume volume-d63ab5c0-3b4d-4b47-abe4-15273a454841 (d63ab5c0-3b4d-4b47-abe4-15273a454841): created successfully >2018-08-22 02:13:51.069 39124 INFO cinder.volume.manager [req-ac80f37f-4588-40c1-8d1d-3c5609c02e46 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Created volume successfully. >2018-08-22 02:14:08.195 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d1ca38bf-6944-4bbe-8ecb-d496b0b95fd4 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] initialize_connection: Did not find a preferred node for volume volume-d63ab5c0-3b4d-4b47-abe4-15273a454841. >2018-08-22 02:14:08.395 39124 INFO cinder.volume.manager [req-d1ca38bf-6944-4bbe-8ecb-d496b0b95fd4 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Initialize volume connection completed successfully. >2018-08-22 02:14:13.161 39124 INFO cinder.volume.manager [req-14107433-a2c4-44d3-a79c-3687c6abf663 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Attaching volume d63ab5c0-3b4d-4b47-abe4-15273a454841 to instance 74c369e5-b61c-4631-a67a-4ed93ee0eb8e at mountpoint /dev/vdb on host None. >2018-08-22 02:14:13.234 39124 INFO cinder.volume.manager [req-14107433-a2c4-44d3-a79c-3687c6abf663 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Attach volume completed successfully. >2018-08-22 02:14:14.956 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f649d944-28df-4fa9-846d-d9174d6cf597 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Unable to get remote copy information for volume volume-d63ab5c0-3b4d-4b47-abe4-15273a454841 >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova [req-f649d944-28df-4fa9-846d-d9174d6cf597 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Failed to notify nova on events: [{'tag': 'd63ab5c0-3b4d-4b47-abe4-15273a454841', 'name': 'volume-extended', 'server_uuid': '74c369e5-b61c-4631-a67a-4ed93ee0eb8e'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-d0066717-f233-4332-9f2a-6f368e8dc043) >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova return_raw=True) >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-d0066717-f233-4332-9f2a-6f368e8dc043) >2018-08-22 02:14:15.693 39124 ERROR cinder.compute.nova >2018-08-22 02:14:15.718 39124 INFO cinder.volume.manager [req-f649d944-28df-4fa9-846d-d9174d6cf597 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Extend volume completed successfully. >2018-08-22 02:19:28.667 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-7f2eaa77-61e1-45f5-a17d-99446eb1e5a3 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:19:28.943 39124 INFO cinder.volume.manager [req-7f2eaa77-61e1-45f5-a17d-99446eb1e5a3 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Terminate volume connection completed successfully. >2018-08-22 02:19:29.332 39124 INFO cinder.volume.manager [req-9cbcdcb4-9295-4f52-85ed-ec7fd4859470 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Detaching volume d63ab5c0-3b4d-4b47-abe4-15273a454841 from instance 74c369e5-b61c-4631-a67a-4ed93ee0eb8e. >2018-08-22 02:19:29.418 39124 INFO cinder.volume.manager [req-9cbcdcb4-9295-4f52-85ed-ec7fd4859470 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Detach volume completed successfully. >2018-08-22 02:19:32.879 39124 INFO cinder.volume.manager [req-18e20d2b-4d11-4b7f-b7f9-d24f15ceb5c0 cb9c23adc6224229be2069343159b74a db4b28396d2d4cd4828d649d0cbbdbc2 - default default] Deleted volume successfully. >2018-08-22 02:19:38.476 39124 INFO cinder.volume.flows.manager.create_volume [req-846b476e-b797-4f19-a801-eb266b30b9e4 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] Volume 7ff88c99-c288-484d-a082-0a4b618ded39: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7ff88c99-c288-484d-a082-0a4b618ded39'} >2018-08-22 02:19:39.005 39124 INFO cinder.volume.flows.manager.create_volume [req-846b476e-b797-4f19-a801-eb266b30b9e4 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] Volume volume-7ff88c99-c288-484d-a082-0a4b618ded39 (7ff88c99-c288-484d-a082-0a4b618ded39): created successfully >2018-08-22 02:19:39.009 39124 INFO cinder.volume.manager [req-846b476e-b797-4f19-a801-eb266b30b9e4 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] Created volume successfully. >2018-08-22 02:19:40.835 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-4156b079-ad95-4cf5-8c86-c269a636d233 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] Unable to get remote copy information for volume volume-7ff88c99-c288-484d-a082-0a4b618ded39 >2018-08-22 02:19:41.120 39124 INFO cinder.volume.manager [req-4156b079-ad95-4cf5-8c86-c269a636d233 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] Extend volume completed successfully. >2018-08-22 02:19:43.877 39124 INFO cinder.volume.manager [req-93dcaa79-dd6e-4645-916d-07c9c0b8e6e0 7f24e455ada4401bac34fa517815fbc1 da5e0452703f4da3ace77db3f17a574c - default default] Deleted volume successfully. >2018-08-22 02:19:48.340 39124 INFO cinder.volume.flows.manager.create_volume [req-19ae1bbe-fd07-42d4-a401-81c9b6dba43e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume bafd4a57-30d1-4abc-a9af-a5b9f81b7ce6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bafd4a57-30d1-4abc-a9af-a5b9f81b7ce6'} >2018-08-22 02:19:48.878 39124 INFO cinder.volume.flows.manager.create_volume [req-19ae1bbe-fd07-42d4-a401-81c9b6dba43e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume volume-bafd4a57-30d1-4abc-a9af-a5b9f81b7ce6 (bafd4a57-30d1-4abc-a9af-a5b9f81b7ce6): created successfully >2018-08-22 02:19:48.882 39124 INFO cinder.volume.manager [req-19ae1bbe-fd07-42d4-a401-81c9b6dba43e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Created volume successfully. >2018-08-22 02:19:51.131 39124 INFO cinder.volume.flows.manager.create_volume [req-b61502bf-0004-4d3e-bba3-82085f3d0a6b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume a3c5ffbe-271a-4e12-a394-d1fe64ca4b66: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a3c5ffbe-271a-4e12-a394-d1fe64ca4b66'} >2018-08-22 02:19:51.671 39124 INFO cinder.volume.flows.manager.create_volume [req-b61502bf-0004-4d3e-bba3-82085f3d0a6b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume volume-a3c5ffbe-271a-4e12-a394-d1fe64ca4b66 (a3c5ffbe-271a-4e12-a394-d1fe64ca4b66): created successfully >2018-08-22 02:19:51.675 39124 INFO cinder.volume.manager [req-b61502bf-0004-4d3e-bba3-82085f3d0a6b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Created volume successfully. >2018-08-22 02:19:57.614 39124 INFO cinder.volume.manager [req-a4d98dc2-5f62-4ab2-9629-ea0fa6d3dc3b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Deleted volume successfully. >2018-08-22 02:19:58.984 39124 INFO cinder.volume.manager [req-ca8aa2f1-8eda-41c6-af12-54077e8157bd e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Deleted volume successfully. >2018-08-22 02:19:59.532 39124 INFO cinder.volume.flows.manager.create_volume [req-c8099ac3-702f-4ce9-a4a9-cf712e9daa9e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume f19dd4ba-2605-485e-aba9-3fb6823c1ece: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f19dd4ba-2605-485e-aba9-3fb6823c1ece'} >2018-08-22 02:20:00.064 39124 INFO cinder.volume.flows.manager.create_volume [req-c8099ac3-702f-4ce9-a4a9-cf712e9daa9e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume volume-f19dd4ba-2605-485e-aba9-3fb6823c1ece (f19dd4ba-2605-485e-aba9-3fb6823c1ece): created successfully >2018-08-22 02:20:00.067 39124 INFO cinder.volume.manager [req-c8099ac3-702f-4ce9-a4a9-cf712e9daa9e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Created volume successfully. >2018-08-22 02:20:00.927 39124 INFO cinder.volume.flows.manager.create_volume [req-5d1eadb0-fb7b-453a-9273-14bee7129ccb e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume bf5488dd-4af1-4cef-9c24-40b2cc8c075d: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bf5488dd-4af1-4cef-9c24-40b2cc8c075d', 'source_volid': 'f19dd4ba-2605-485e-aba9-3fb6823c1ece', 'source_volstatus': u'available'} >2018-08-22 02:20:02.455 39124 INFO cinder.volume.flows.manager.create_volume [req-5d1eadb0-fb7b-453a-9273-14bee7129ccb e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume volume-bf5488dd-4af1-4cef-9c24-40b2cc8c075d (bf5488dd-4af1-4cef-9c24-40b2cc8c075d): created successfully >2018-08-22 02:20:02.459 39124 INFO cinder.volume.manager [req-5d1eadb0-fb7b-453a-9273-14bee7129ccb e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Created volume successfully. >2018-08-22 02:20:03.950 39124 INFO cinder.volume.flows.manager.create_volume [req-51af013c-4c1d-460f-962f-315dfbfdc36b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume 6a11ef1a-95bd-40d9-8192-a781285b9b4f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6a11ef1a-95bd-40d9-8192-a781285b9b4f'} >2018-08-22 02:20:04.485 39124 INFO cinder.volume.flows.manager.create_volume [req-51af013c-4c1d-460f-962f-315dfbfdc36b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume volume-6a11ef1a-95bd-40d9-8192-a781285b9b4f (6a11ef1a-95bd-40d9-8192-a781285b9b4f): created successfully >2018-08-22 02:20:04.489 39124 INFO cinder.volume.manager [req-51af013c-4c1d-460f-962f-315dfbfdc36b e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Created volume successfully. >2018-08-22 02:20:06.449 39124 INFO cinder.volume.manager [req-5ad19b3e-e892-4c52-99fc-4d79d8ab3312 e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Deleted volume successfully. >2018-08-22 02:20:11.017 39124 INFO cinder.volume.manager [req-4db91464-0153-4fa2-9f57-e9cce61769d6 e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Deleted volume successfully. >2018-08-22 02:20:12.027 39124 INFO cinder.volume.flows.manager.create_volume [req-872eb4b0-7206-49c3-a5f5-973e0616a4ba e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume 7d025dc7-33ec-451d-983f-10d7d7db9658: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-7d025dc7-33ec-451d-983f-10d7d7db9658', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b247add0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 02:20:13.638 39124 INFO cinder.image.image_utils [req-872eb4b0-7206-49c3-a5f5-973e0616a4ba e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Image download 12.00 MB at 7.46 MB/s >2018-08-22 02:20:23.569 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 02:20:28.680 39124 INFO cinder.image.image_utils [req-872eb4b0-7206-49c3-a5f5-973e0616a4ba e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 02:20:30.533 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-872eb4b0-7206-49c3-a5f5-973e0616a4ba e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:20:30.877 39124 INFO cinder.volume.flows.manager.create_volume [req-872eb4b0-7206-49c3-a5f5-973e0616a4ba e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume volume-7d025dc7-33ec-451d-983f-10d7d7db9658 (7d025dc7-33ec-451d-983f-10d7d7db9658): created successfully >2018-08-22 02:20:30.881 39124 INFO cinder.volume.manager [req-872eb4b0-7206-49c3-a5f5-973e0616a4ba e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Created volume successfully. >2018-08-22 02:20:31.966 39124 INFO cinder.volume.flows.manager.create_volume [req-3cfdcc3d-33cb-44a6-a1a4-4d2c6ef50efa e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume 9a847a64-61f6-4888-a221-047f7b1360aa: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9a847a64-61f6-4888-a221-047f7b1360aa'} >2018-08-22 02:20:32.527 39124 INFO cinder.volume.flows.manager.create_volume [req-3cfdcc3d-33cb-44a6-a1a4-4d2c6ef50efa e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Volume volume-9a847a64-61f6-4888-a221-047f7b1360aa (9a847a64-61f6-4888-a221-047f7b1360aa): created successfully >2018-08-22 02:20:32.531 39124 INFO cinder.volume.manager [req-3cfdcc3d-33cb-44a6-a1a4-4d2c6ef50efa e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Created volume successfully. >2018-08-22 02:20:34.268 39124 INFO cinder.volume.manager [req-965b08a3-5cd9-43c5-ba16-5cdea27e224e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Deleted volume successfully. >2018-08-22 02:20:35.462 39124 INFO cinder.volume.manager [req-2c66e45e-a33a-40bf-98e5-0f695b31db9e e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Deleted volume successfully. >2018-08-22 02:20:36.674 39124 INFO cinder.volume.manager [req-589c468f-6e62-46e0-9727-c3029c56aee3 e8a0035346b64d52b15ab536e6abdabc 3a22783bb4604fda83f141ea89836823 - default default] Deleted volume successfully. >2018-08-22 02:20:45.587 39124 INFO cinder.volume.flows.manager.create_volume [req-888f49ea-84fe-470b-aef0-866d75e9d695 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Volume 91def83e-0c94-40a1-a26c-513ce50e60e0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-91def83e-0c94-40a1-a26c-513ce50e60e0'} >2018-08-22 02:20:46.129 39124 INFO cinder.volume.flows.manager.create_volume [req-888f49ea-84fe-470b-aef0-866d75e9d695 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Volume volume-91def83e-0c94-40a1-a26c-513ce50e60e0 (91def83e-0c94-40a1-a26c-513ce50e60e0): created successfully >2018-08-22 02:20:46.133 39124 INFO cinder.volume.manager [req-888f49ea-84fe-470b-aef0-866d75e9d695 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Created volume successfully. >2018-08-22 02:20:48.134 39124 INFO cinder.volume.flows.manager.create_volume [req-4c4470cd-3819-4262-aab1-626631711cd1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Volume fbb45bdb-743a-4e43-b1ab-b9f76f9ede0a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fbb45bdb-743a-4e43-b1ab-b9f76f9ede0a'} >2018-08-22 02:20:48.667 39124 INFO cinder.volume.flows.manager.create_volume [req-4c4470cd-3819-4262-aab1-626631711cd1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Volume volume-fbb45bdb-743a-4e43-b1ab-b9f76f9ede0a (fbb45bdb-743a-4e43-b1ab-b9f76f9ede0a): created successfully >2018-08-22 02:20:48.671 39124 INFO cinder.volume.manager [req-4c4470cd-3819-4262-aab1-626631711cd1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Created volume successfully. >2018-08-22 02:20:50.136 39124 INFO cinder.volume.flows.manager.create_volume [req-cf9b5749-d75b-4295-a0be-7d1319ba4ca1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Volume 4e134cac-a79b-450b-991f-21092386dd1a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4e134cac-a79b-450b-991f-21092386dd1a'} >2018-08-22 02:20:50.674 39124 INFO cinder.volume.flows.manager.create_volume [req-cf9b5749-d75b-4295-a0be-7d1319ba4ca1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Volume volume-4e134cac-a79b-450b-991f-21092386dd1a (4e134cac-a79b-450b-991f-21092386dd1a): created successfully >2018-08-22 02:20:50.680 39124 INFO cinder.volume.manager [req-cf9b5749-d75b-4295-a0be-7d1319ba4ca1 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Created volume successfully. >2018-08-22 02:20:57.079 39124 INFO cinder.volume.manager [req-86867b95-2c21-434f-b9ef-de2647eaa355 caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Deleted volume successfully. >2018-08-22 02:20:57.099 39124 INFO cinder.volume.manager [req-e197b40b-2ee4-42f2-81ac-082b56d8d1ee caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Deleted volume successfully. >2018-08-22 02:20:57.327 39124 INFO cinder.volume.manager [req-ec344f59-d322-412e-a0c5-b3e7239b383f caf9b85ed4784ae1bb31083abff613a7 878f6aeb6f154bc5ac1222602ac70db1 - default default] Deleted volume successfully. >2018-08-22 02:21:02.462 39124 INFO cinder.volume.flows.manager.create_volume [req-9fb2765c-c8d1-437b-b766-20c358e99c47 74251f18a60a488eb8052a615913c8fa 1a97211b0ae845c0a8675e8f4e72cf6f - default default] Volume da8fa3ce-b4ed-4471-89fd-3f7a50d58f8f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-da8fa3ce-b4ed-4471-89fd-3f7a50d58f8f'} >2018-08-22 02:21:02.997 39124 INFO cinder.volume.flows.manager.create_volume [req-9fb2765c-c8d1-437b-b766-20c358e99c47 74251f18a60a488eb8052a615913c8fa 1a97211b0ae845c0a8675e8f4e72cf6f - default default] Volume volume-da8fa3ce-b4ed-4471-89fd-3f7a50d58f8f (da8fa3ce-b4ed-4471-89fd-3f7a50d58f8f): created successfully >2018-08-22 02:21:03.001 39124 INFO cinder.volume.manager [req-9fb2765c-c8d1-437b-b766-20c358e99c47 74251f18a60a488eb8052a615913c8fa 1a97211b0ae845c0a8675e8f4e72cf6f - default default] Created volume successfully. >2018-08-22 02:21:24.510 39124 INFO cinder.volume.manager [req-2cbb82a3-a42b-4078-a56f-6587ce78f5a4 74251f18a60a488eb8052a615913c8fa 1a97211b0ae845c0a8675e8f4e72cf6f - default default] Deleted volume successfully. >2018-08-22 02:21:28.877 39124 INFO cinder.volume.flows.manager.create_volume [req-5a0a69ff-0a26-4445-9a50-18906bdb95c5 5c17b7b528ba42d7927f3d6b19c86ebf 50e1b3ac37e54d6f976136ac7a7b5f41 - default default] Volume 289e1464-5e51-448c-a571-83da058b191c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-289e1464-5e51-448c-a571-83da058b191c'} >2018-08-22 02:21:32.921 39124 INFO cinder.volume.flows.manager.create_volume [req-5a0a69ff-0a26-4445-9a50-18906bdb95c5 5c17b7b528ba42d7927f3d6b19c86ebf 50e1b3ac37e54d6f976136ac7a7b5f41 - default default] Volume volume-289e1464-5e51-448c-a571-83da058b191c (289e1464-5e51-448c-a571-83da058b191c): created successfully >2018-08-22 02:21:32.925 39124 INFO cinder.volume.manager [req-5a0a69ff-0a26-4445-9a50-18906bdb95c5 5c17b7b528ba42d7927f3d6b19c86ebf 50e1b3ac37e54d6f976136ac7a7b5f41 - default default] Created volume successfully. >2018-08-22 02:21:36.486 39124 INFO cinder.volume.manager [req-fb04de5a-3ebd-41c5-9099-b4b43a511dc5 5c17b7b528ba42d7927f3d6b19c86ebf 50e1b3ac37e54d6f976136ac7a7b5f41 - default default] Deleted volume successfully. >2018-08-22 02:22:24.905 39124 INFO cinder.volume.flows.manager.create_volume [req-fc3cb6d3-cd17-4d57-bd0d-3533df820ede 7888447054544c78adb745668439f035 9442727785024726964f556f17ffd563 - default default] Volume 4c80410c-0301-41a4-a834-e8003a33c880: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4c80410c-0301-41a4-a834-e8003a33c880'} >2018-08-22 02:22:25.450 39124 INFO cinder.volume.flows.manager.create_volume [req-fc3cb6d3-cd17-4d57-bd0d-3533df820ede 7888447054544c78adb745668439f035 9442727785024726964f556f17ffd563 - default default] Volume volume-4c80410c-0301-41a4-a834-e8003a33c880 (4c80410c-0301-41a4-a834-e8003a33c880): created successfully >2018-08-22 02:22:25.454 39124 INFO cinder.volume.manager [req-fc3cb6d3-cd17-4d57-bd0d-3533df820ede 7888447054544c78adb745668439f035 9442727785024726964f556f17ffd563 - default default] Created volume successfully. >2018-08-22 02:22:28.661 39124 INFO cinder.volume.manager [req-c76357b9-9c05-4cd5-bf48-ed78bdc711f0 7888447054544c78adb745668439f035 9442727785024726964f556f17ffd563 - default default] Deleted volume successfully. >2018-08-22 02:22:34.238 39124 INFO cinder.volume.flows.manager.create_volume [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Volume dc23b67d-5daf-481c-9e07-3b28512937d4: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-dc23b67d-5daf-481c-9e07-3b28512937d4', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b1b4b190>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 02:22:35.142 39124 INFO cinder.image.image_utils [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 02:22:44.620 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] initialize_connection: Did not find a preferred node for volume volume-dc23b67d-5daf-481c-9e07-3b28512937d4. >2018-08-22 02:22:45.256 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 02:22:50.332 39124 INFO cinder.image.image_utils [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 02:22:52.536 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:22:53.011 39124 INFO cinder.volume.flows.manager.create_volume [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Volume volume-dc23b67d-5daf-481c-9e07-3b28512937d4 (dc23b67d-5daf-481c-9e07-3b28512937d4): created successfully >2018-08-22 02:22:53.015 39124 INFO cinder.volume.manager [req-7614a468-d52f-476f-a943-9640d121c7d5 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Created volume successfully. >2018-08-22 02:22:54.270 39124 INFO cinder.volume.flows.manager.create_volume [req-3491a474-d18c-43ac-85fa-70f9623e2f7c 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Volume 02622438-f7b7-4330-9511-15202e7bdc95: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-02622438-f7b7-4330-9511-15202e7bdc95', 'source_volid': 'dc23b67d-5daf-481c-9e07-3b28512937d4', 'source_volstatus': u'available'} >2018-08-22 02:22:55.748 39124 INFO cinder.volume.flows.manager.create_volume [req-3491a474-d18c-43ac-85fa-70f9623e2f7c 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Volume volume-02622438-f7b7-4330-9511-15202e7bdc95 (02622438-f7b7-4330-9511-15202e7bdc95): created successfully >2018-08-22 02:22:55.752 39124 INFO cinder.volume.manager [req-3491a474-d18c-43ac-85fa-70f9623e2f7c 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Created volume successfully. >2018-08-22 02:22:59.707 39124 INFO cinder.volume.manager [req-1f4a5a8f-2688-4247-bc16-6f393d96fd7b 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Deleted volume successfully. >2018-08-22 02:23:00.986 39124 INFO cinder.volume.manager [req-dc057dc8-85a0-4ce8-986a-d66b5dde5699 4dd7fe331c9546f8b6bdb94fd181565a da551665809b47dc892fb74579c5749c - default default] Deleted volume successfully. >2018-08-22 02:23:06.098 39124 INFO cinder.volume.flows.manager.create_volume [req-6f328098-f9b3-4f9a-972d-e52316693d2e add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Volume 9ffc1378-340f-427b-9287-bb44106cfc1f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9ffc1378-340f-427b-9287-bb44106cfc1f'} >2018-08-22 02:23:06.623 39124 INFO cinder.volume.flows.manager.create_volume [req-6f328098-f9b3-4f9a-972d-e52316693d2e add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Volume volume-9ffc1378-340f-427b-9287-bb44106cfc1f (9ffc1378-340f-427b-9287-bb44106cfc1f): created successfully >2018-08-22 02:23:06.627 39124 INFO cinder.volume.manager [req-6f328098-f9b3-4f9a-972d-e52316693d2e add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Created volume successfully. >2018-08-22 02:23:08.370 39124 INFO cinder.volume.flows.manager.create_volume [req-5be3f0d2-6f3d-4d86-935d-40caf15eb6d2 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Volume 739bf2c9-558b-4299-8982-73c78d544396: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-739bf2c9-558b-4299-8982-73c78d544396'} >2018-08-22 02:23:08.896 39124 INFO cinder.volume.flows.manager.create_volume [req-5be3f0d2-6f3d-4d86-935d-40caf15eb6d2 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Volume volume-739bf2c9-558b-4299-8982-73c78d544396 (739bf2c9-558b-4299-8982-73c78d544396): created successfully >2018-08-22 02:23:08.899 39124 INFO cinder.volume.manager [req-5be3f0d2-6f3d-4d86-935d-40caf15eb6d2 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Created volume successfully. >2018-08-22 02:23:10.480 39116 INFO cinder.volume.flows.manager.create_volume [req-c6bded51-8901-4054-ada0-420e0ebad9d6 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Volume ff70702d-186b-4a87-90d3-1e2c56a04fbb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ff70702d-186b-4a87-90d3-1e2c56a04fbb'} >2018-08-22 02:23:11.044 39116 INFO cinder.volume.flows.manager.create_volume [req-c6bded51-8901-4054-ada0-420e0ebad9d6 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Volume volume-ff70702d-186b-4a87-90d3-1e2c56a04fbb (ff70702d-186b-4a87-90d3-1e2c56a04fbb): created successfully >2018-08-22 02:23:11.048 39116 INFO cinder.volume.manager [req-c6bded51-8901-4054-ada0-420e0ebad9d6 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Created volume successfully. >2018-08-22 02:23:14.313 39124 INFO cinder.volume.manager [req-fd548fbb-02d8-499d-9eb4-992c0a83d5f3 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Deleted volume successfully. >2018-08-22 02:23:14.555 39124 INFO cinder.volume.manager [req-e0653797-5f95-4599-8068-f25c44e87f08 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Deleted volume successfully. >2018-08-22 02:23:14.785 39116 INFO cinder.volume.manager [req-3a86ca30-f06f-4f77-ad9e-df690e7b9196 add4f2837ea64866a3d3b5db15d32d96 f97df4f5007844f0b1fcd9067dfd4166 - default default] Deleted volume successfully. >2018-08-22 02:23:34.416 39116 INFO cinder.volume.flows.manager.create_volume [req-c007ab23-e3eb-47e0-8fd0-61692170290a 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Volume 9584e979-4669-411c-b537-f1fa81753409: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9584e979-4669-411c-b537-f1fa81753409'} >2018-08-22 02:23:34.964 39116 INFO cinder.volume.flows.manager.create_volume [req-c007ab23-e3eb-47e0-8fd0-61692170290a 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Volume volume-9584e979-4669-411c-b537-f1fa81753409 (9584e979-4669-411c-b537-f1fa81753409): created successfully >2018-08-22 02:23:34.968 39116 INFO cinder.volume.manager [req-c007ab23-e3eb-47e0-8fd0-61692170290a 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Created volume successfully. >2018-08-22 02:23:36.370 39124 INFO cinder.volume.flows.manager.create_volume [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Volume e4abf915-5d12-4b27-98a2-7d5aa867d1f4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4'} >2018-08-22 02:23:36.914 39124 INFO cinder.volume.flows.manager.create_volume [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Volume volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4 (e4abf915-5d12-4b27-98a2-7d5aa867d1f4): created successfully >2018-08-22 02:23:36.918 39124 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Created volume successfully. >2018-08-22 02:23:44.567 39124 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Initialize volume connection completed successfully. >2018-08-22 02:23:44.959 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 02:23:48.241 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] initialize_connection: Did not find a preferred node for volume volume-9584e979-4669-411c-b537-f1fa81753409. >2018-08-22 02:23:48.509 39116 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Initialize volume connection completed successfully. >2018-08-22 02:23:48.896 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 02:23:54.027 39116 INFO cinder.volume.utils [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Volume copy 1024.00 MB at 467.29 MB/s >2018-08-22 02:23:56.537 39124 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Terminate volume connection completed successfully. >2018-08-22 02:23:56.578 39124 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Remove volume export completed successfully. >2018-08-22 02:23:58.428 39116 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:23:58.701 39116 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Terminate volume connection completed successfully. >2018-08-22 02:23:58.733 39116 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Remove volume export completed successfully. >2018-08-22 02:23:58.978 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 02:24:03.530 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Error running SSH command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 02:24:03.532 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 02:24:03.532 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Error running SSH command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 02:24:03.533 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-9584e979-4669-411c-b537-f1fa81753409', '"volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9584e979-4669-411c-b537-f1fa81753409 "volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 02:24:03.534 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Unable to rename the logical volume for volume: 9584e979-4669-411c-b537-f1fa81753409: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-9584e979-4669-411c-b537-f1fa81753409', '"volume-e4abf915-5d12-4b27-98a2-7d5aa867d1f4"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-22 02:24:03.683 39116 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Complete-Migrate volume completed successfully. >2018-08-22 02:24:03.684 39116 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Migrate volume completed successfully. >2018-08-22 02:24:04.211 39116 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Retype volume completed successfully. >2018-08-22 02:24:04.953 39116 INFO cinder.volume.manager [req-3cf8079a-b604-4a0f-8eca-7434ec6f2c17 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Deleted volume successfully. >2018-08-22 02:24:07.203 39124 WARNING cinder.quota [req-2d93eaa1-6cfb-42b0-a296-2968c838aacf 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1556570105 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1556570105, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 02:24:07.204 39124 WARNING cinder.quota [req-2d93eaa1-6cfb-42b0-a296-2968c838aacf 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1556570105 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1556570105, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 02:24:07.459 39124 INFO cinder.volume.manager [req-2d93eaa1-6cfb-42b0-a296-2968c838aacf 5f351f2def9440dd8823e37ba9465cf2 dc968b7529e7478c804188ec898da71e - default default] Deleted volume successfully. >2018-08-22 02:24:28.760 39124 INFO cinder.volume.flows.manager.create_volume [req-7940c1a0-9c6f-4b43-845d-04cde3fdf2c0 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Volume 8bd31ad9-1ae8-4388-a767-296683f54e2a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8bd31ad9-1ae8-4388-a767-296683f54e2a'} >2018-08-22 02:24:29.295 39124 INFO cinder.volume.flows.manager.create_volume [req-7940c1a0-9c6f-4b43-845d-04cde3fdf2c0 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Volume volume-8bd31ad9-1ae8-4388-a767-296683f54e2a (8bd31ad9-1ae8-4388-a767-296683f54e2a): created successfully >2018-08-22 02:24:29.299 39124 INFO cinder.volume.manager [req-7940c1a0-9c6f-4b43-845d-04cde3fdf2c0 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Created volume successfully. >2018-08-22 02:24:32.096 39124 INFO cinder.volume.manager [req-70e2b47d-bee4-45ef-b1e6-29f5a5b5730e e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Create snapshot completed successfully >2018-08-22 02:24:35.426 39124 INFO cinder.volume.manager [req-8bac2e79-9085-4321-910f-4f32f15958d4 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Create snapshot completed successfully >2018-08-22 02:24:38.550 39124 INFO cinder.volume.manager [req-3c925c8e-1b98-47b0-8661-a229c7ce8196 649da464b9244d2c949eeb2c556bf219 4101b241126f4293b809935931dc23e1 - default default] Delete snapshot completed successfully. >2018-08-22 02:24:44.753 39124 INFO cinder.volume.manager [req-c0bce64d-c140-4f43-9879-8b0f5d04e818 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Create snapshot completed successfully >2018-08-22 02:24:47.484 39124 INFO cinder.volume.manager [req-c2f816ef-2f62-485a-ba51-55e128e0ab5d 649da464b9244d2c949eeb2c556bf219 4101b241126f4293b809935931dc23e1 - default default] Delete snapshot completed successfully. >2018-08-22 02:24:49.906 39124 INFO cinder.volume.manager [req-350adac0-3545-4f12-8dc9-32b7868d6935 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Create snapshot completed successfully >2018-08-22 02:24:52.815 39124 INFO cinder.volume.manager [req-269748de-3abe-43c4-ad73-f171aec7c482 649da464b9244d2c949eeb2c556bf219 4101b241126f4293b809935931dc23e1 - default default] Delete snapshot completed successfully. >2018-08-22 02:24:55.595 39124 INFO cinder.volume.manager [req-250c1d18-2e16-47fa-b14f-53cf6b875873 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Create snapshot completed successfully >2018-08-22 02:24:58.707 39124 INFO cinder.volume.manager [req-50590263-299b-4092-b06f-4afbb0c90da3 649da464b9244d2c949eeb2c556bf219 4101b241126f4293b809935931dc23e1 - default default] Delete snapshot completed successfully. >2018-08-22 02:25:01.415 39124 INFO cinder.volume.manager [req-11ccf429-3b5f-4bd9-8804-f5df53aae3f6 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Delete snapshot completed successfully. >2018-08-22 02:25:03.198 39124 INFO cinder.volume.manager [req-ed0b6495-9e51-49d8-833e-416c9248f4c5 e1aea0683fcb41caa7e23120cb54ab1f b8365796ba35479892a706aa5dcb75d1 - default default] Deleted volume successfully. >2018-08-22 02:25:08.106 39124 INFO cinder.volume.flows.manager.create_volume [req-73e11564-1ac7-4135-911d-9f3b09ebecbb bedb312c825f4b0e8ee83ec76c232dd0 ed8340b880474e96a081427aadd2be43 - default default] Volume b53e7fc3-0ac6-4cc7-91bd-b8a180685166: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b53e7fc3-0ac6-4cc7-91bd-b8a180685166'} >2018-08-22 02:25:08.632 39124 INFO cinder.volume.flows.manager.create_volume [req-73e11564-1ac7-4135-911d-9f3b09ebecbb bedb312c825f4b0e8ee83ec76c232dd0 ed8340b880474e96a081427aadd2be43 - default default] Volume volume-b53e7fc3-0ac6-4cc7-91bd-b8a180685166 (b53e7fc3-0ac6-4cc7-91bd-b8a180685166): created successfully >2018-08-22 02:25:08.636 39124 INFO cinder.volume.manager [req-73e11564-1ac7-4135-911d-9f3b09ebecbb bedb312c825f4b0e8ee83ec76c232dd0 ed8340b880474e96a081427aadd2be43 - default default] Created volume successfully. >2018-08-22 02:25:11.582 39124 INFO cinder.volume.manager [req-a824334b-ccea-4eff-b928-5a81916b38fc bedb312c825f4b0e8ee83ec76c232dd0 ed8340b880474e96a081427aadd2be43 - default default] Create snapshot completed successfully >2018-08-22 02:25:16.707 39124 INFO cinder.volume.manager [req-314a6a7f-1e43-4234-aae1-ab518e4aa7e8 bedb312c825f4b0e8ee83ec76c232dd0 ed8340b880474e96a081427aadd2be43 - default default] Delete snapshot completed successfully. >2018-08-22 02:25:18.916 39124 INFO cinder.volume.manager [req-b232c324-b57c-4a1a-b328-99cabfb443fd bedb312c825f4b0e8ee83ec76c232dd0 ed8340b880474e96a081427aadd2be43 - default default] Deleted volume successfully. >2018-08-22 02:25:23.737 39124 INFO cinder.volume.flows.manager.create_volume [req-4f23811b-e625-43cc-a27c-b2767e97ddae 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Volume 65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6'} >2018-08-22 02:25:24.545 39124 INFO cinder.volume.flows.manager.create_volume [req-4f23811b-e625-43cc-a27c-b2767e97ddae 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Volume volume-65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6 (65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6): created successfully >2018-08-22 02:25:24.549 39124 INFO cinder.volume.manager [req-4f23811b-e625-43cc-a27c-b2767e97ddae 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Created volume successfully. >2018-08-22 02:25:28.532 39124 INFO cinder.volume.manager [req-f4be970a-1a5e-495b-828d-c5de283ffaed 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:25:44.563 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c5a2338d-acaa-401e-947a-eb34e4a8e421 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] initialize_connection: Did not find a preferred node for volume volume-8977186e-a8eb-4284-ad9c-a0c112163c9f. >2018-08-22 02:25:44.994 39124 INFO cinder.volume.manager [req-c5a2338d-acaa-401e-947a-eb34e4a8e421 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Initialize volume connection completed successfully. >2018-08-22 02:26:07.170 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c5a2338d-acaa-401e-947a-eb34e4a8e421 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:26:07.468 39124 INFO cinder.volume.manager [req-c5a2338d-acaa-401e-947a-eb34e4a8e421 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Terminate volume connection completed successfully. >2018-08-22 02:26:07.510 39124 INFO cinder.volume.manager [req-c5a2338d-acaa-401e-947a-eb34e4a8e421 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Remove volume export completed successfully. >2018-08-22 02:26:08.767 39124 INFO cinder.volume.manager [req-c5a2338d-acaa-401e-947a-eb34e4a8e421 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Deleted volume successfully. >2018-08-22 02:26:24.729 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-39dcf740-07bc-4965-9c0d-ddaf19376f43 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] initialize_connection: Did not find a preferred node for volume volume-65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6. >2018-08-22 02:26:24.930 39124 INFO cinder.volume.manager [req-39dcf740-07bc-4965-9c0d-ddaf19376f43 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Initialize volume connection completed successfully. >2018-08-22 02:26:29.426 39124 INFO cinder.volume.manager [req-cec072e1-35ea-4e37-83d5-fd1afac96000 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Attaching volume 65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6 to instance 800c6ef3-18fe-4485-b538-4bb36a7b88dc at mountpoint /dev/vdb on host None. >2018-08-22 02:26:29.506 39124 INFO cinder.volume.manager [req-cec072e1-35ea-4e37-83d5-fd1afac96000 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Attach volume completed successfully. >2018-08-22 02:26:31.525 39124 INFO cinder.volume.manager [req-11ad6841-ae5f-4574-bf4c-d41fbd3c8c82 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:26:33.912 39124 INFO cinder.volume.manager [req-0e0a4593-96c6-4891-86a4-37fbb61c261a 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:26:36.099 39124 INFO cinder.volume.manager [req-ec878487-d916-429c-819d-3e76e60a65fa 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:26:38.553 39124 INFO cinder.volume.manager [req-806baf6b-827e-4618-a625-74ddd1b246be 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:26:40.755 39124 INFO cinder.volume.manager [req-1ddea4b3-c7b0-4734-bf42-2bde14021874 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:26:43.027 39124 INFO cinder.volume.manager [req-c061e7fa-7809-4241-a9bb-f14dc22f84e0 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:26:54.091 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-5c15adaa-0cc8-4f75-b7ca-edd678f4b43f 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:26:54.366 39124 INFO cinder.volume.manager [req-5c15adaa-0cc8-4f75-b7ca-edd678f4b43f 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Terminate volume connection completed successfully. >2018-08-22 02:26:54.512 39124 INFO cinder.volume.manager [req-9566c095-dc4f-472b-87d1-6b881d463203 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Detaching volume 65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6 from instance 800c6ef3-18fe-4485-b538-4bb36a7b88dc. >2018-08-22 02:26:54.598 39124 INFO cinder.volume.manager [req-9566c095-dc4f-472b-87d1-6b881d463203 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Detach volume completed successfully. >2018-08-22 02:26:58.944 39124 INFO cinder.volume.manager [req-e14e7814-c2bd-4b4d-bf47-8eb17b2d60c3 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:27:01.961 39124 INFO cinder.volume.manager [req-822814b8-128a-4e95-add4-c0467c634938 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:27:04.021 39124 INFO cinder.volume.manager [req-c1cadd2a-8d5a-4b01-ae17-ba2e9c73f310 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:27:19.569 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d65e1340-ef2c-4738-8f30-46221448f8c4 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] initialize_connection: Did not find a preferred node for volume volume-65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6. >2018-08-22 02:27:19.813 39124 INFO cinder.volume.manager [req-d65e1340-ef2c-4738-8f30-46221448f8c4 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Initialize volume connection completed successfully. >2018-08-22 02:27:24.313 39124 INFO cinder.volume.manager [req-8bd9947f-7aac-49d3-9e8c-9608425ee496 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Attaching volume 65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6 to instance d027a88d-9a9c-4722-96fe-900676db2d25 at mountpoint /dev/vdb on host None. >2018-08-22 02:27:24.385 39124 INFO cinder.volume.manager [req-8bd9947f-7aac-49d3-9e8c-9608425ee496 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Attach volume completed successfully. >2018-08-22 02:27:27.249 39124 INFO cinder.volume.manager [req-c823899d-ccb5-4534-80ac-c9a7f7e0538f 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:27:29.735 39124 INFO cinder.volume.manager [req-76e0b0bb-03cf-4b67-908b-3a459a44ebe0 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:27:31.873 39124 INFO cinder.volume.manager [req-744aae6d-bf98-4bea-87e3-950b2a8a0a91 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:27:34.088 39124 INFO cinder.volume.manager [req-23265967-6663-46f7-b197-519d0af758f3 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:27:36.307 39124 INFO cinder.volume.manager [req-ddee1e2c-d257-4096-9ee2-b31436687374 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:27:46.680 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-47521c58-fcdc-4aec-84c2-fbd748cc9b97 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:27:46.956 39124 INFO cinder.volume.manager [req-47521c58-fcdc-4aec-84c2-fbd748cc9b97 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Terminate volume connection completed successfully. >2018-08-22 02:27:47.099 39124 INFO cinder.volume.manager [req-426c68ba-26ce-4650-a88e-7128a24eb607 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Detaching volume 65e8787c-b223-4dd1-b7e9-c6ebcd3ba7b6 from instance d027a88d-9a9c-4722-96fe-900676db2d25. >2018-08-22 02:27:47.186 39124 INFO cinder.volume.manager [req-426c68ba-26ce-4650-a88e-7128a24eb607 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Detach volume completed successfully. >2018-08-22 02:27:50.131 39124 INFO cinder.volume.flows.manager.create_volume [req-254873d5-fb33-4725-995d-505bf2e1fc44 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Volume df04419d-62ff-4533-bf35-e6221bee83fd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-df04419d-62ff-4533-bf35-e6221bee83fd'} >2018-08-22 02:27:50.662 39124 INFO cinder.volume.flows.manager.create_volume [req-254873d5-fb33-4725-995d-505bf2e1fc44 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Volume volume-df04419d-62ff-4533-bf35-e6221bee83fd (df04419d-62ff-4533-bf35-e6221bee83fd): created successfully >2018-08-22 02:27:50.666 39124 INFO cinder.volume.manager [req-254873d5-fb33-4725-995d-505bf2e1fc44 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Created volume successfully. >2018-08-22 02:27:53.148 39124 INFO cinder.volume.manager [req-bdb3b2dd-a022-4f1d-94a8-a1896874ac4c 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Create snapshot completed successfully >2018-08-22 02:27:53.744 39124 INFO cinder.volume.flows.manager.create_volume [req-b66765dc-713f-47a1-aaf4-0bd0da141dda 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Volume b644ae39-db92-43e2-8420-80c13dcceaa8: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': '62154953-9fa7-4c93-afd7-1f19fec8fbdf', 'volume_name': 'volume-b644ae39-db92-43e2-8420-80c13dcceaa8'} >2018-08-22 02:28:53.055 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-b66765dc-713f-47a1-aaf4-0bd0da141dda 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Unable to get remote copy information for volume volume-b644ae39-db92-43e2-8420-80c13dcceaa8 >2018-08-22 02:28:53.365 39124 INFO cinder.volume.flows.manager.create_volume [req-b66765dc-713f-47a1-aaf4-0bd0da141dda 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Volume volume-b644ae39-db92-43e2-8420-80c13dcceaa8 (b644ae39-db92-43e2-8420-80c13dcceaa8): created successfully >2018-08-22 02:28:53.369 39124 INFO cinder.volume.manager [req-b66765dc-713f-47a1-aaf4-0bd0da141dda 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Created volume successfully. >2018-08-22 02:28:54.795 39124 INFO cinder.volume.manager [req-c5996d2b-01c7-4ce2-994b-55dd2552aca0 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Deleted volume successfully. >2018-08-22 02:28:58.318 39124 INFO cinder.volume.manager [req-60c5792c-3918-4388-81e9-83bbd47edbec 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:28:58.710 39124 INFO cinder.volume.manager [req-1d3b3f6e-3a23-4b7b-94b3-239ef620a2e2 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Delete snapshot completed successfully. >2018-08-22 02:29:01.193 39124 WARNING cinder.db.sqlalchemy.api [req-3b1b6ad9-7091-409a-8c7d-4793e4dc02e4 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-22 02:29:01.226 39124 INFO cinder.volume.manager [req-96a73f94-2397-46e8-bea2-58a7407813ed 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Deleted volume successfully. >2018-08-22 02:29:01.474 39124 INFO cinder.volume.manager [req-3b1b6ad9-7091-409a-8c7d-4793e4dc02e4 3b68cd230ec54ce881eee9a43bf49710 6d240f9efd1d4ea7afc55a0fbae11013 - default default] Deleted volume successfully. >2018-08-22 02:29:06.505 39124 INFO cinder.volume.flows.manager.create_volume [req-4239f6e4-1631-49f9-a8db-46937461fa8c 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Volume fd72e0a5-1960-4180-b6d9-ac791d020c28: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fd72e0a5-1960-4180-b6d9-ac791d020c28'} >2018-08-22 02:29:07.036 39124 INFO cinder.volume.flows.manager.create_volume [req-4239f6e4-1631-49f9-a8db-46937461fa8c 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Volume volume-fd72e0a5-1960-4180-b6d9-ac791d020c28 (fd72e0a5-1960-4180-b6d9-ac791d020c28): created successfully >2018-08-22 02:29:07.040 39124 INFO cinder.volume.manager [req-4239f6e4-1631-49f9-a8db-46937461fa8c 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Created volume successfully. >2018-08-22 02:29:10.106 39124 INFO cinder.volume.manager [req-540343c6-d668-448f-92e6-a4d611a8d50b 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Create snapshot completed successfully >2018-08-22 02:29:13.054 39124 INFO cinder.volume.manager [req-5ad5e231-c530-4274-9d66-6022a3dfd959 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Create snapshot completed successfully >2018-08-22 02:29:17.260 39124 INFO cinder.volume.manager [req-be790bd5-e2b6-4537-8ee1-b856701ba33d 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Create snapshot completed successfully >2018-08-22 02:29:24.662 39124 INFO cinder.volume.manager [req-c8da66c9-ece6-446f-b179-5ad34af89112 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Delete snapshot completed successfully. >2018-08-22 02:29:24.887 39124 INFO cinder.volume.manager [req-bb61973f-6a50-4c8f-83e9-2c762002da56 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Delete snapshot completed successfully. >2018-08-22 02:29:25.101 39124 INFO cinder.volume.manager [req-b9ff2eb4-867c-484d-9b6e-64106dd8dd50 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Delete snapshot completed successfully. >2018-08-22 02:29:27.142 39124 INFO cinder.volume.manager [req-45e083cf-9323-4fd1-98c9-b25fd91e79a7 33b87ff909804d63ba70c6c8813617b8 89edf262ef6c428b9062b102c1394b6a - default default] Deleted volume successfully. >2018-08-22 02:29:32.523 39124 INFO cinder.volume.flows.manager.create_volume [req-e02348a1-375c-4368-aea2-4ea1d876dc77 8790680223754625985040d7507c6d13 34c68e0880e44614b967332cc6bab8d4 - default default] Volume 0c6d8f59-af72-4ffa-a44f-f5c1b9abe3a8: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-0c6d8f59-af72-4ffa-a44f-f5c1b9abe3a8'} >2018-08-22 02:29:33.475 39124 INFO cinder.volume.flows.manager.create_volume [req-e02348a1-375c-4368-aea2-4ea1d876dc77 8790680223754625985040d7507c6d13 34c68e0880e44614b967332cc6bab8d4 - default default] Volume volume-0c6d8f59-af72-4ffa-a44f-f5c1b9abe3a8 (0c6d8f59-af72-4ffa-a44f-f5c1b9abe3a8): created successfully >2018-08-22 02:29:33.479 39124 INFO cinder.volume.manager [req-e02348a1-375c-4368-aea2-4ea1d876dc77 8790680223754625985040d7507c6d13 34c68e0880e44614b967332cc6bab8d4 - default default] Created volume successfully. >2018-08-22 02:29:36.944 39124 INFO cinder.volume.manager [req-9047a518-815f-406d-9ad3-b4d80de97a1f 8790680223754625985040d7507c6d13 34c68e0880e44614b967332cc6bab8d4 - default default] Create snapshot completed successfully >2018-08-22 02:29:40.951 39124 INFO cinder.volume.manager [req-eda94174-1714-42da-bc64-27f5613da50a 8790680223754625985040d7507c6d13 34c68e0880e44614b967332cc6bab8d4 - default default] Delete snapshot completed successfully. >2018-08-22 02:29:44.354 39124 INFO cinder.volume.manager [req-edada702-77a6-4c3b-b390-543836dcde6e 8790680223754625985040d7507c6d13 34c68e0880e44614b967332cc6bab8d4 - default default] Deleted volume successfully. >2018-08-22 02:30:07.424 39124 INFO cinder.volume.flows.manager.create_volume [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume 009abcd9-47ba-453d-a17e-b9045ce26baa: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-009abcd9-47ba-453d-a17e-b9045ce26baa', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b1d79550>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 02:30:08.259 39124 INFO cinder.image.image_utils [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 02:30:17.899 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] initialize_connection: Did not find a preferred node for volume volume-009abcd9-47ba-453d-a17e-b9045ce26baa. >2018-08-22 02:30:18.510 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 02:30:23.637 39124 INFO cinder.image.image_utils [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 02:30:27.773 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 02:30:28.121 39124 INFO cinder.volume.flows.manager.create_volume [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume volume-009abcd9-47ba-453d-a17e-b9045ce26baa (009abcd9-47ba-453d-a17e-b9045ce26baa): created successfully >2018-08-22 02:30:28.125 39124 INFO cinder.volume.manager [req-c7526d20-74c7-4124-af08-2f062a92f568 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Created volume successfully. >2018-08-22 02:30:29.583 39124 INFO cinder.volume.flows.manager.create_volume [req-648d7980-3927-4755-b287-effa63f1003c 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume 69757e1d-235f-4391-ab05-92c0229d72d0: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-69757e1d-235f-4391-ab05-92c0229d72d0', 'source_volid': '009abcd9-47ba-453d-a17e-b9045ce26baa', 'source_volstatus': u'available'} >2018-08-22 02:30:31.321 39124 INFO cinder.volume.flows.manager.create_volume [req-648d7980-3927-4755-b287-effa63f1003c 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume volume-69757e1d-235f-4391-ab05-92c0229d72d0 (69757e1d-235f-4391-ab05-92c0229d72d0): created successfully >2018-08-22 02:30:31.325 39124 INFO cinder.volume.manager [req-648d7980-3927-4755-b287-effa63f1003c 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Created volume successfully. >2018-08-22 02:30:32.339 39124 INFO cinder.volume.flows.manager.create_volume [req-4e96b88c-7ac8-4c61-9bde-208bf4c046da 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume 891a16b8-dc97-4e7d-9c60-2b70f6919c2a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-891a16b8-dc97-4e7d-9c60-2b70f6919c2a'} >2018-08-22 02:30:32.868 39124 INFO cinder.volume.flows.manager.create_volume [req-4e96b88c-7ac8-4c61-9bde-208bf4c046da 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume volume-891a16b8-dc97-4e7d-9c60-2b70f6919c2a (891a16b8-dc97-4e7d-9c60-2b70f6919c2a): created successfully >2018-08-22 02:30:32.872 39124 INFO cinder.volume.manager [req-4e96b88c-7ac8-4c61-9bde-208bf4c046da 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Created volume successfully. >2018-08-22 02:30:33.931 39124 INFO cinder.volume.flows.manager.create_volume [req-934815bb-9f14-4456-9179-1ff3cc9404d0 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume 65db596c-5ec2-4049-b8a4-6b92f4b74a1b: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-65db596c-5ec2-4049-b8a4-6b92f4b74a1b', 'source_volid': '891a16b8-dc97-4e7d-9c60-2b70f6919c2a', 'source_volstatus': u'available'} >2018-08-22 02:30:51.172 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-934815bb-9f14-4456-9179-1ff3cc9404d0 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Unable to get remote copy information for volume volume-65db596c-5ec2-4049-b8a4-6b92f4b74a1b >2018-08-22 02:30:51.551 39124 INFO cinder.volume.flows.manager.create_volume [req-934815bb-9f14-4456-9179-1ff3cc9404d0 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Volume volume-65db596c-5ec2-4049-b8a4-6b92f4b74a1b (65db596c-5ec2-4049-b8a4-6b92f4b74a1b): created successfully >2018-08-22 02:30:51.555 39124 INFO cinder.volume.manager [req-934815bb-9f14-4456-9179-1ff3cc9404d0 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Created volume successfully. >2018-08-22 02:30:52.662 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 02:30:52.756 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 02:30:52.762 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 02:30:52.855 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 02:30:55.703 39124 INFO cinder.volume.manager [req-57b4f426-8278-4f28-a66b-17351ea89730 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Deleted volume successfully. >2018-08-22 02:30:55.725 39124 INFO cinder.volume.manager [req-0a8b6a3c-9699-4890-adb4-32857cbcc06d 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Deleted volume successfully. >2018-08-22 02:30:55.816 39124 INFO cinder.volume.manager [req-d9cfd967-47df-47ab-9fed-8c20f9b63664 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Deleted volume successfully. >2018-08-22 02:30:56.000 39124 INFO cinder.volume.manager [req-d7b2c68a-60bc-4c18-bb03-78c25fc972a4 9008906bd23a4161a38b815484f59993 9dd2ca72a70945ea8f96c44784a59c5d - default default] Deleted volume successfully. >2018-08-22 02:31:00.396 39124 INFO cinder.volume.flows.manager.create_volume [req-5c46cb39-686b-48f5-a0b4-68776d83af7a 77e96ec2e315471da7dc288ba62ba30c 35c3caf16d02498c86cb4cb562f1e7c8 - default default] Volume 97b0a330-1222-4819-b1cc-455cbcb250ca: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-97b0a330-1222-4819-b1cc-455cbcb250ca'} >2018-08-22 02:31:01.235 39124 INFO cinder.volume.flows.manager.create_volume [req-5c46cb39-686b-48f5-a0b4-68776d83af7a 77e96ec2e315471da7dc288ba62ba30c 35c3caf16d02498c86cb4cb562f1e7c8 - default default] Volume volume-97b0a330-1222-4819-b1cc-455cbcb250ca (97b0a330-1222-4819-b1cc-455cbcb250ca): created successfully >2018-08-22 02:31:01.239 39124 INFO cinder.volume.manager [req-5c46cb39-686b-48f5-a0b4-68776d83af7a 77e96ec2e315471da7dc288ba62ba30c 35c3caf16d02498c86cb4cb562f1e7c8 - default default] Created volume successfully. >2018-08-22 02:31:03.520 39124 INFO cinder.volume.manager [req-475d8a03-ea2d-453c-852f-431b2a19235f 77e96ec2e315471da7dc288ba62ba30c 35c3caf16d02498c86cb4cb562f1e7c8 - default default] Deleted volume successfully. >2018-08-22 02:31:26.985 39124 INFO cinder.volume.flows.manager.create_volume [req-7d16419f-2107-4300-a5d4-90fbbe1f34ff e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] Volume 084053d3-2484-4ecd-affa-5ebd94ede633: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-084053d3-2484-4ecd-affa-5ebd94ede633'} >2018-08-22 02:31:27.588 39124 INFO cinder.volume.flows.manager.create_volume [req-7d16419f-2107-4300-a5d4-90fbbe1f34ff e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] Volume volume-084053d3-2484-4ecd-affa-5ebd94ede633 (084053d3-2484-4ecd-affa-5ebd94ede633): created successfully >2018-08-22 02:31:27.592 39124 INFO cinder.volume.manager [req-7d16419f-2107-4300-a5d4-90fbbe1f34ff e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] Created volume successfully. >2018-08-22 02:31:30.607 39124 INFO cinder.volume.manager [req-aa2f794a-03ef-4a6e-9630-09dbefae4ecb e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] Deleted volume successfully. >2018-08-22 02:31:31.123 39124 INFO cinder.volume.flows.manager.create_volume [req-39134417-02f5-42f6-94e3-09df573590d6 e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] Volume 4041db2d-9767-4acb-877b-77d95103a2dc: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4041db2d-9767-4acb-877b-77d95103a2dc'} >2018-08-22 02:31:31.653 39124 INFO cinder.volume.flows.manager.create_volume [req-39134417-02f5-42f6-94e3-09df573590d6 e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] Volume volume-4041db2d-9767-4acb-877b-77d95103a2dc (4041db2d-9767-4acb-877b-77d95103a2dc): created successfully >2018-08-22 02:31:31.657 39124 INFO cinder.volume.manager [req-39134417-02f5-42f6-94e3-09df573590d6 e7da6ea2fcbd433b825710a21304bdcf 6a08817f01274043bf2d5a5310fc5578 - default default] Created volume successfully. >2018-08-22 02:31:33.430 39124 INFO cinder.volume.manager [req-831bde07-f32e-4d42-8580-e28aa23543d9 7eefd9bb9e65412faef290bbd9b40f6d 3748497657ed4b77b741a4a83c5aac65 - default default] Transfer volume completed successfully. >2018-08-22 02:31:35.244 39124 INFO cinder.volume.manager [req-d8a65cd3-6c6a-4b92-ab1e-5b54be487a00 dea9947a3d5c451fa660ea879e863a10 c44d8d4b0fda453787c79dc3a25ec727 - default default] Deleted volume successfully. >2018-08-22 02:31:44.877 39124 INFO cinder.volume.flows.manager.create_volume [req-944c96f2-0d84-4d6c-8a83-415f3b528eb7 2990117964514594910c5d13a531e071 a29756953da74347bf4d6cbc8e1dbddc - default default] Volume dbdc8a0d-deb6-41b2-b7c8-8c0c65279781: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-dbdc8a0d-deb6-41b2-b7c8-8c0c65279781'} >2018-08-22 02:31:45.407 39124 INFO cinder.volume.flows.manager.create_volume [req-944c96f2-0d84-4d6c-8a83-415f3b528eb7 2990117964514594910c5d13a531e071 a29756953da74347bf4d6cbc8e1dbddc - default default] Volume volume-dbdc8a0d-deb6-41b2-b7c8-8c0c65279781 (dbdc8a0d-deb6-41b2-b7c8-8c0c65279781): created successfully >2018-08-22 02:31:45.411 39124 INFO cinder.volume.manager [req-944c96f2-0d84-4d6c-8a83-415f3b528eb7 2990117964514594910c5d13a531e071 a29756953da74347bf4d6cbc8e1dbddc - default default] Created volume successfully. >2018-08-22 02:31:49.216 39124 INFO cinder.volume.manager [req-8984a985-641d-48ba-9dac-2ef42c00470c 2990117964514594910c5d13a531e071 a29756953da74347bf4d6cbc8e1dbddc - default default] Deleted volume successfully. >2018-08-22 02:31:56.043 39124 INFO cinder.volume.flows.manager.create_volume [req-88fd6586-78e0-4b14-8e35-9c36d9458d1f c0322baf0c3644bc9a135ab2794dd0c2 bdabb12d9a06470f84d4a6433a8aefb0 - default default] Volume 94a974b3-88b5-448e-bbf5-f2c003ad2c1f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-94a974b3-88b5-448e-bbf5-f2c003ad2c1f'} >2018-08-22 02:31:56.584 39124 INFO cinder.volume.flows.manager.create_volume [req-88fd6586-78e0-4b14-8e35-9c36d9458d1f c0322baf0c3644bc9a135ab2794dd0c2 bdabb12d9a06470f84d4a6433a8aefb0 - default default] Volume volume-94a974b3-88b5-448e-bbf5-f2c003ad2c1f (94a974b3-88b5-448e-bbf5-f2c003ad2c1f): created successfully >2018-08-22 02:31:56.588 39124 INFO cinder.volume.manager [req-88fd6586-78e0-4b14-8e35-9c36d9458d1f c0322baf0c3644bc9a135ab2794dd0c2 bdabb12d9a06470f84d4a6433a8aefb0 - default default] Created volume successfully. >2018-08-22 02:31:59.194 39124 INFO cinder.volume.manager [req-04f63ca5-91ee-44a5-9416-e0079baa1acf c0322baf0c3644bc9a135ab2794dd0c2 bdabb12d9a06470f84d4a6433a8aefb0 - default default] Create snapshot completed successfully >2018-08-22 02:32:03.379 39124 INFO cinder.volume.manager [req-01b42786-221a-4f0f-904c-9f710984f2b2 c0322baf0c3644bc9a135ab2794dd0c2 bdabb12d9a06470f84d4a6433a8aefb0 - default default] Delete snapshot completed successfully. >2018-08-22 02:32:05.322 39124 INFO cinder.volume.manager [req-80403d48-552f-42d2-90c7-654fb41fa6f7 c0322baf0c3644bc9a135ab2794dd0c2 bdabb12d9a06470f84d4a6433a8aefb0 - default default] Deleted volume successfully. >2018-08-22 03:15:09.222 39124 INFO cinder.volume.manager [req-6d7295f0-4ddc-4793-9c3b-74fb46794b89 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-2017489343: creating >2018-08-22 03:15:09.247 39124 INFO cinder.volume.manager [req-6d7295f0-4ddc-4793-9c3b-74fb46794b89 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-2017489343: created successfully >2018-08-22 03:15:09.270 39124 INFO cinder.volume.manager [req-6d7295f0-4ddc-4793-9c3b-74fb46794b89 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Create group completed successfully. >2018-08-22 03:15:09.980 39124 INFO cinder.volume.flows.manager.create_volume [req-204541b8-da76-47f5-859e-ebdc1c26d02b 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume 9c7edc60-8162-49a2-a75f-c5294e68a327: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9c7edc60-8162-49a2-a75f-c5294e68a327'} >2018-08-22 03:15:10.719 39124 INFO cinder.volume.flows.manager.create_volume [req-204541b8-da76-47f5-859e-ebdc1c26d02b 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume volume-9c7edc60-8162-49a2-a75f-c5294e68a327 (9c7edc60-8162-49a2-a75f-c5294e68a327): created successfully >2018-08-22 03:15:10.723 39124 INFO cinder.volume.manager [req-204541b8-da76-47f5-859e-ebdc1c26d02b 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Created volume successfully. >2018-08-22 03:15:12.565 39124 INFO cinder.volume.manager [req-7df1a54f-3d9a-4fd8-83ed-1ae3dc88cbe5 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] GroupSnapshot 607dcab9-52f0-4450-8cc2-03689b849933: creating. >2018-08-22 03:15:16.974 39124 INFO cinder.volume.manager [req-7df1a54f-3d9a-4fd8-83ed-1ae3dc88cbe5 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] group_snapshot 607dcab9-52f0-4450-8cc2-03689b849933: created successfully >2018-08-22 03:15:17.200 39124 INFO cinder.volume.manager [req-217ae8ac-6026-4b09-a6d9-7dc0a7b2aa92 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] group_snapshot 607dcab9-52f0-4450-8cc2-03689b849933: deleting >2018-08-22 03:15:19.330 39124 WARNING cinder.quota [req-217ae8ac-6026-4b09-a6d9-7dc0a7b2aa92 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-191434194 is set by the default quota flag: quota_snapshots_tempest-volume-type-191434194, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:19.364 39124 INFO cinder.volume.manager [req-217ae8ac-6026-4b09-a6d9-7dc0a7b2aa92 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] group_snapshot 607dcab9-52f0-4450-8cc2-03689b849933: deleted successfully >2018-08-22 03:15:21.472 39124 WARNING cinder.quota [req-60986908-642f-4b28-9014-0fbc6a1b76bd 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-191434194 is set by the default quota flag: quota_gigabytes_tempest-volume-type-191434194, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:21.473 39124 WARNING cinder.quota [req-60986908-642f-4b28-9014-0fbc6a1b76bd 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-191434194 is set by the default quota flag: quota_volumes_tempest-volume-type-191434194, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:21.742 39124 INFO cinder.volume.manager [req-60986908-642f-4b28-9014-0fbc6a1b76bd 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Delete group completed successfully. >2018-08-22 03:15:24.129 39124 INFO cinder.volume.manager [req-dff8deaa-7b09-441c-ac65-201572e6a58e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-133320864: creating >2018-08-22 03:15:24.150 39124 INFO cinder.volume.manager [req-dff8deaa-7b09-441c-ac65-201572e6a58e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-133320864: created successfully >2018-08-22 03:15:24.172 39124 INFO cinder.volume.manager [req-dff8deaa-7b09-441c-ac65-201572e6a58e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Create group completed successfully. >2018-08-22 03:15:24.635 39124 INFO cinder.volume.flows.manager.create_volume [req-8d08b42f-f494-4894-bd9d-f2c1e75a2b12 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume 6445f5fb-f05f-4043-84a0-6f62cc372856: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6445f5fb-f05f-4043-84a0-6f62cc372856'} >2018-08-22 03:15:25.462 39124 INFO cinder.volume.flows.manager.create_volume [req-8d08b42f-f494-4894-bd9d-f2c1e75a2b12 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume volume-6445f5fb-f05f-4043-84a0-6f62cc372856 (6445f5fb-f05f-4043-84a0-6f62cc372856): created successfully >2018-08-22 03:15:25.467 39124 INFO cinder.volume.manager [req-8d08b42f-f494-4894-bd9d-f2c1e75a2b12 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Created volume successfully. >2018-08-22 03:15:26.753 39124 WARNING cinder.quota [req-ca6daba8-afca-4475-8d36-09ee0a476e57 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-90154043 is set by the default quota flag: quota_volumes_tempest-volume-type-90154043, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:26.754 39124 WARNING cinder.quota [req-ca6daba8-afca-4475-8d36-09ee0a476e57 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-90154043 is set by the default quota flag: quota_gigabytes_tempest-volume-type-90154043, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:27.030 39124 INFO cinder.volume.manager [req-ca6daba8-afca-4475-8d36-09ee0a476e57 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Delete group completed successfully. >2018-08-22 03:15:27.362 39124 INFO cinder.volume.manager [req-1da9c28e-e514-4da8-92f8-2ef8cb823e95 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-484822130: creating >2018-08-22 03:15:27.383 39124 INFO cinder.volume.manager [req-1da9c28e-e514-4da8-92f8-2ef8cb823e95 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-484822130: created successfully >2018-08-22 03:15:27.404 39124 INFO cinder.volume.manager [req-1da9c28e-e514-4da8-92f8-2ef8cb823e95 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Create group completed successfully. >2018-08-22 03:15:27.753 39124 INFO cinder.volume.flows.manager.create_volume [req-6ca4bea2-f556-418d-adf2-54280acae600 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume 688e9bbb-5e81-4afb-a849-287f7430b949: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-688e9bbb-5e81-4afb-a849-287f7430b949'} >2018-08-22 03:15:28.364 39124 INFO cinder.volume.flows.manager.create_volume [req-6ca4bea2-f556-418d-adf2-54280acae600 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume volume-688e9bbb-5e81-4afb-a849-287f7430b949 (688e9bbb-5e81-4afb-a849-287f7430b949): created successfully >2018-08-22 03:15:28.368 39124 INFO cinder.volume.manager [req-6ca4bea2-f556-418d-adf2-54280acae600 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Created volume successfully. >2018-08-22 03:15:28.994 39124 INFO cinder.volume.manager [req-5f136ef6-c6bb-4e5e-92b3-e79ea2b23528 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] GroupSnapshot 32bae89a-008b-42b7-8c7d-53ca0c2baa1a: creating. >2018-08-22 03:15:31.127 39124 INFO cinder.volume.manager [req-5f136ef6-c6bb-4e5e-92b3-e79ea2b23528 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] group_snapshot 32bae89a-008b-42b7-8c7d-53ca0c2baa1a: created successfully >2018-08-22 03:15:36.399 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-be79606b-1360-4293-89c1-c7ef12a4772c 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Updating status for CG: 02bbf6ff-c984-4197-91d2-21fced52b984. >2018-08-22 03:15:36.504 39124 INFO cinder.volume.manager [req-be79606b-1360-4293-89c1-c7ef12a4772c 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Create group from source-snapshot-32bae89a-008b-42b7-8c7d-53ca0c2baa1a completed successfully. >2018-08-22 03:15:41.597 39124 WARNING cinder.quota [req-0ac3cbe0-f4ab-412c-b6d6-7d2404719aa8 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-256437276 is set by the default quota flag: quota_volumes_tempest-volume-type-256437276, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:41.599 39124 WARNING cinder.quota [req-0ac3cbe0-f4ab-412c-b6d6-7d2404719aa8 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-256437276 is set by the default quota flag: quota_gigabytes_tempest-volume-type-256437276, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:41.871 39124 INFO cinder.volume.manager [req-0ac3cbe0-f4ab-412c-b6d6-7d2404719aa8 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Delete group completed successfully. >2018-08-22 03:15:42.202 39124 INFO cinder.volume.manager [req-d5f6a5e2-e0e9-403a-b5b8-1e4507b5eea9 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] group_snapshot 32bae89a-008b-42b7-8c7d-53ca0c2baa1a: deleting >2018-08-22 03:15:44.389 39124 WARNING cinder.quota [req-d5f6a5e2-e0e9-403a-b5b8-1e4507b5eea9 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-256437276 is set by the default quota flag: quota_snapshots_tempest-volume-type-256437276, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:44.420 39124 INFO cinder.volume.manager [req-d5f6a5e2-e0e9-403a-b5b8-1e4507b5eea9 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] group_snapshot 32bae89a-008b-42b7-8c7d-53ca0c2baa1a: deleted successfully >2018-08-22 03:15:46.230 39124 INFO cinder.volume.manager [req-1dc38988-fc2f-4862-9052-eba370e46759 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Delete group completed successfully. >2018-08-22 03:15:47.273 39124 INFO cinder.volume.manager [req-05fc13b3-6652-43a5-9b8e-9ecf0335da85 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-761519177: creating >2018-08-22 03:15:47.295 39124 INFO cinder.volume.manager [req-05fc13b3-6652-43a5-9b8e-9ecf0335da85 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Group tempest-CG-761519177: created successfully >2018-08-22 03:15:47.314 39124 INFO cinder.volume.manager [req-05fc13b3-6652-43a5-9b8e-9ecf0335da85 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Create group completed successfully. >2018-08-22 03:15:47.654 39124 INFO cinder.volume.flows.manager.create_volume [req-fda1daf0-c89a-41e2-bfe8-935bcac8ddd7 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume 178f2160-6fab-4e2b-adc1-627494b0aed5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-178f2160-6fab-4e2b-adc1-627494b0aed5'} >2018-08-22 03:15:48.206 39124 INFO cinder.volume.flows.manager.create_volume [req-fda1daf0-c89a-41e2-bfe8-935bcac8ddd7 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Volume volume-178f2160-6fab-4e2b-adc1-627494b0aed5 (178f2160-6fab-4e2b-adc1-627494b0aed5): created successfully >2018-08-22 03:15:48.210 39124 INFO cinder.volume.manager [req-fda1daf0-c89a-41e2-bfe8-935bcac8ddd7 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Created volume successfully. >2018-08-22 03:15:54.550 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1bd89366-eefa-4fb7-ab9b-346245992ac6 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Updating status for CG: 06afd731-9c5d-40b5-a762-c8513adf8939. >2018-08-22 03:15:54.637 39124 INFO cinder.volume.manager [req-1bd89366-eefa-4fb7-ab9b-346245992ac6 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Create group from source-group-4a131fcd-d4e9-401a-afb3-0a4ff8200fcd completed successfully. >2018-08-22 03:15:59.204 39124 WARNING cinder.quota [req-a1369d76-4376-4149-9cea-e7e6bccd574e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-730953180 is set by the default quota flag: quota_volumes_tempest-volume-type-730953180, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:59.205 39124 WARNING cinder.quota [req-a1369d76-4376-4149-9cea-e7e6bccd574e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-730953180 is set by the default quota flag: quota_gigabytes_tempest-volume-type-730953180, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:15:59.538 39124 INFO cinder.volume.manager [req-a1369d76-4376-4149-9cea-e7e6bccd574e 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Delete group completed successfully. >2018-08-22 03:16:00.901 39124 INFO cinder.volume.manager [req-6386bc27-2153-49fe-b133-1881ea384925 76f125e0ad0b427d9bd3ea1ab5db54d4 be6b7e3f181b4b8b8da9140189d3fb7a - default default] Delete group completed successfully. >2018-08-22 03:17:09.700 39124 INFO cinder.volume.flows.manager.create_volume [req-6d06a6eb-d0ac-4e4d-a45c-82cb273592e5 a0660fa26f9f4fabade9fa4b3b36505c 9cccb8dcddce4df280a06c5b5aaa5277 - default default] Volume d22c8420-0819-4e66-ab49-5b93c9016006: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d22c8420-0819-4e66-ab49-5b93c9016006'} >2018-08-22 03:17:10.230 39124 INFO cinder.volume.flows.manager.create_volume [req-6d06a6eb-d0ac-4e4d-a45c-82cb273592e5 a0660fa26f9f4fabade9fa4b3b36505c 9cccb8dcddce4df280a06c5b5aaa5277 - default default] Volume volume-d22c8420-0819-4e66-ab49-5b93c9016006 (d22c8420-0819-4e66-ab49-5b93c9016006): created successfully >2018-08-22 03:17:10.234 39124 INFO cinder.volume.manager [req-6d06a6eb-d0ac-4e4d-a45c-82cb273592e5 a0660fa26f9f4fabade9fa4b3b36505c 9cccb8dcddce4df280a06c5b5aaa5277 - default default] Created volume successfully. >2018-08-22 03:17:12.829 39124 INFO cinder.volume.manager [req-52039f9a-3893-4fd5-8654-7eacebac179a a0660fa26f9f4fabade9fa4b3b36505c 9cccb8dcddce4df280a06c5b5aaa5277 - default default] Deleted volume successfully. >2018-08-22 03:17:21.796 39124 INFO cinder.volume.flows.manager.create_volume [req-2d2670fd-4290-47bd-bec2-2c2da2ec477a 975f2057dfa9409694ed410c8fd3092b 635058ac850a423a9952ede526669af3 - default default] Volume caad9ef7-7f00-401e-8640-026ce957d57d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-caad9ef7-7f00-401e-8640-026ce957d57d'} >2018-08-22 03:17:22.415 39124 INFO cinder.volume.flows.manager.create_volume [req-2d2670fd-4290-47bd-bec2-2c2da2ec477a 975f2057dfa9409694ed410c8fd3092b 635058ac850a423a9952ede526669af3 - default default] Volume volume-caad9ef7-7f00-401e-8640-026ce957d57d (caad9ef7-7f00-401e-8640-026ce957d57d): created successfully >2018-08-22 03:17:22.419 39124 INFO cinder.volume.manager [req-2d2670fd-4290-47bd-bec2-2c2da2ec477a 975f2057dfa9409694ed410c8fd3092b 635058ac850a423a9952ede526669af3 - default default] Created volume successfully. >2018-08-22 03:17:25.665 39124 WARNING cinder.quota [req-e53987ef-d5c7-4bf6-b4b2-719efab7a062 975f2057dfa9409694ed410c8fd3092b 635058ac850a423a9952ede526669af3 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1320849203 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1320849203, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:17:25.666 39124 WARNING cinder.quota [req-e53987ef-d5c7-4bf6-b4b2-719efab7a062 975f2057dfa9409694ed410c8fd3092b 635058ac850a423a9952ede526669af3 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1320849203 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1320849203, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:17:25.917 39124 INFO cinder.volume.manager [req-e53987ef-d5c7-4bf6-b4b2-719efab7a062 975f2057dfa9409694ed410c8fd3092b 635058ac850a423a9952ede526669af3 - default default] Deleted volume successfully. >2018-08-22 03:17:34.398 39124 INFO cinder.volume.flows.manager.create_volume [req-a9e5e822-cbde-4e97-88ad-b3233fe91ee2 f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] Volume d00d6428-3af0-48ae-bbb4-9a1777797628: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d00d6428-3af0-48ae-bbb4-9a1777797628'} >2018-08-22 03:17:34.952 39124 INFO cinder.volume.flows.manager.create_volume [req-a9e5e822-cbde-4e97-88ad-b3233fe91ee2 f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] Volume volume-d00d6428-3af0-48ae-bbb4-9a1777797628 (d00d6428-3af0-48ae-bbb4-9a1777797628): created successfully >2018-08-22 03:17:34.957 39124 INFO cinder.volume.manager [req-a9e5e822-cbde-4e97-88ad-b3233fe91ee2 f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] Created volume successfully. >2018-08-22 03:17:36.522 39124 INFO cinder.volume.manager [req-752dadd6-7598-48e9-8dfb-3ee8f9cf77ed f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] Retype volume completed successfully. >2018-08-22 03:17:40.475 39124 WARNING cinder.quota [req-c0173935-9e43-4da2-b669-539080c55e74 f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-265654424 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-265654424, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:17:40.476 39124 WARNING cinder.quota [req-c0173935-9e43-4da2-b669-539080c55e74 f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-265654424 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-265654424, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:17:40.792 39124 INFO cinder.volume.manager [req-c0173935-9e43-4da2-b669-539080c55e74 f0d3705108664d57a5eab5a4d95e1231 6a8747b929cf4a41bb5fa14a8d24038c - default default] Deleted volume successfully. >2018-08-22 03:18:22.887 39124 INFO cinder.volume.flows.manager.create_volume [req-1a43a1ae-b249-4426-8ee1-b08619632cb1 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume 3dc8b9e5-4ae3-44a4-aebb-8bc694e3873c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3dc8b9e5-4ae3-44a4-aebb-8bc694e3873c'} >2018-08-22 03:18:23.468 39124 INFO cinder.volume.flows.manager.create_volume [req-1a43a1ae-b249-4426-8ee1-b08619632cb1 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume volume-3dc8b9e5-4ae3-44a4-aebb-8bc694e3873c (3dc8b9e5-4ae3-44a4-aebb-8bc694e3873c): created successfully >2018-08-22 03:18:23.472 39124 INFO cinder.volume.manager [req-1a43a1ae-b249-4426-8ee1-b08619632cb1 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Created volume successfully. >2018-08-22 03:18:24.438 39124 INFO cinder.volume.manager [req-be3762c4-cd35-42f0-9a3d-6a9ed805340d 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Attaching volume 3dc8b9e5-4ae3-44a4-aebb-8bc694e3873c to instance 3621cd65-6bbc-45e9-8cec-9e66ef79bb03 at mountpoint /dev/vdb on host None. >2018-08-22 03:18:24.508 39124 INFO cinder.volume.manager [req-be3762c4-cd35-42f0-9a3d-6a9ed805340d 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Attach volume completed successfully. >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager [req-a512c967-b8d2-4d60-a5db-abcd1ff36b5b 4655cd30d7c14df89a582fa41c6a8ce3 4ad21e627b3144b9a29fcb428e59238d - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager force=force) >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-22 03:18:25.968 39124 ERROR cinder.volume.manager >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server [req-a512c967-b8d2-4d60-a5db-abcd1ff36b5b 4655cd30d7c14df89a582fa41c6a8ce3 4ad21e627b3144b9a29fcb428e59238d - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 03:18:25.970 39124 ERROR oslo_messaging.rpc.server >2018-08-22 03:18:26.345 39124 INFO cinder.volume.manager [req-74a23f4d-fe5c-4cdb-b3f9-b0f10a67aa90 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Detaching volume 3dc8b9e5-4ae3-44a4-aebb-8bc694e3873c from instance 3621cd65-6bbc-45e9-8cec-9e66ef79bb03. >2018-08-22 03:18:26.427 39124 INFO cinder.volume.manager [req-74a23f4d-fe5c-4cdb-b3f9-b0f10a67aa90 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Detach volume completed successfully. >2018-08-22 03:18:28.532 39124 INFO cinder.volume.flows.manager.create_volume [req-8928e75e-24c5-49d9-ae7f-e2151d4a8223 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume ec20ede9-afd1-403c-b43f-39ed7b01a860: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ec20ede9-afd1-403c-b43f-39ed7b01a860'} >2018-08-22 03:18:29.057 39124 INFO cinder.volume.flows.manager.create_volume [req-8928e75e-24c5-49d9-ae7f-e2151d4a8223 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume volume-ec20ede9-afd1-403c-b43f-39ed7b01a860 (ec20ede9-afd1-403c-b43f-39ed7b01a860): created successfully >2018-08-22 03:18:29.061 39124 INFO cinder.volume.manager [req-8928e75e-24c5-49d9-ae7f-e2151d4a8223 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Created volume successfully. >2018-08-22 03:18:31.245 39124 INFO cinder.volume.manager [req-ab5f3cf0-60a4-4ca5-8e6d-967446bb62a4 4655cd30d7c14df89a582fa41c6a8ce3 4ad21e627b3144b9a29fcb428e59238d - default default] Deleted volume successfully. >2018-08-22 03:18:31.603 39124 INFO cinder.volume.flows.manager.create_volume [req-8460074a-d158-4d1b-bc90-5872ba907959 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume 5a16a385-53c2-45f3-9532-a76549e8b5d5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5a16a385-53c2-45f3-9532-a76549e8b5d5'} >2018-08-22 03:18:32.178 39124 INFO cinder.volume.flows.manager.create_volume [req-8460074a-d158-4d1b-bc90-5872ba907959 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume volume-5a16a385-53c2-45f3-9532-a76549e8b5d5 (5a16a385-53c2-45f3-9532-a76549e8b5d5): created successfully >2018-08-22 03:18:32.182 39124 INFO cinder.volume.manager [req-8460074a-d158-4d1b-bc90-5872ba907959 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Created volume successfully. >2018-08-22 03:18:34.725 39124 INFO cinder.volume.manager [req-c9e9946d-5a1c-409b-bdf1-2a5a337042a9 4655cd30d7c14df89a582fa41c6a8ce3 4ad21e627b3144b9a29fcb428e59238d - default default] Deleted volume successfully. >2018-08-22 03:18:34.886 39124 INFO cinder.volume.flows.manager.create_volume [req-0b723b2b-b8f5-4d9b-b670-7f71a086bf10 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume 94385f5a-868c-416d-96d1-b74fcf85f457: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-94385f5a-868c-416d-96d1-b74fcf85f457'} >2018-08-22 03:18:35.412 39124 INFO cinder.volume.flows.manager.create_volume [req-0b723b2b-b8f5-4d9b-b670-7f71a086bf10 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume volume-94385f5a-868c-416d-96d1-b74fcf85f457 (94385f5a-868c-416d-96d1-b74fcf85f457): created successfully >2018-08-22 03:18:35.417 39124 INFO cinder.volume.manager [req-0b723b2b-b8f5-4d9b-b670-7f71a086bf10 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Created volume successfully. >2018-08-22 03:18:37.867 39124 INFO cinder.volume.manager [req-8c471e66-7524-4138-87ff-8e5f349c79c4 4655cd30d7c14df89a582fa41c6a8ce3 4ad21e627b3144b9a29fcb428e59238d - default default] Deleted volume successfully. >2018-08-22 03:18:38.215 39124 INFO cinder.volume.flows.manager.create_volume [req-65965b0f-2b6f-45dc-a7fb-ecf618c499e2 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume 316e067a-4a40-43fe-9648-b3e2247f55e1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-316e067a-4a40-43fe-9648-b3e2247f55e1'} >2018-08-22 03:18:38.745 39124 INFO cinder.volume.flows.manager.create_volume [req-65965b0f-2b6f-45dc-a7fb-ecf618c499e2 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume volume-316e067a-4a40-43fe-9648-b3e2247f55e1 (316e067a-4a40-43fe-9648-b3e2247f55e1): created successfully >2018-08-22 03:18:38.748 39124 INFO cinder.volume.manager [req-65965b0f-2b6f-45dc-a7fb-ecf618c499e2 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Created volume successfully. >2018-08-22 03:18:40.640 39124 INFO cinder.volume.manager [req-ffa46615-6f62-4c74-8484-3f4092503ba1 4655cd30d7c14df89a582fa41c6a8ce3 4ad21e627b3144b9a29fcb428e59238d - default default] Deleted volume successfully. >2018-08-22 03:18:40.969 39124 INFO cinder.volume.flows.manager.create_volume [req-4fb40f68-b88e-47ec-96e0-7b032be83315 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume 1cd5a7d6-0142-4024-afb2-92f02bc038b5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1cd5a7d6-0142-4024-afb2-92f02bc038b5'} >2018-08-22 03:18:41.497 39124 INFO cinder.volume.flows.manager.create_volume [req-4fb40f68-b88e-47ec-96e0-7b032be83315 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Volume volume-1cd5a7d6-0142-4024-afb2-92f02bc038b5 (1cd5a7d6-0142-4024-afb2-92f02bc038b5): created successfully >2018-08-22 03:18:41.500 39124 INFO cinder.volume.manager [req-4fb40f68-b88e-47ec-96e0-7b032be83315 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Created volume successfully. >2018-08-22 03:18:44.954 39124 INFO cinder.volume.manager [req-58f2f5ec-227c-47ce-b24d-4e04b1f0fbc0 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Deleted volume successfully. >2018-08-22 03:18:45.198 39124 INFO cinder.volume.manager [req-b2d9845d-cd57-4ba4-9815-e2bf8df75d15 7c34190e73d84ab29e14b0a03c4f86db cb18fccb896246dfb60d21d7aa3a56d1 - default default] Deleted volume successfully. >2018-08-22 03:18:52.187 39124 INFO cinder.volume.flows.manager.create_volume [req-49a9ec28-e566-41da-bb20-ae3545b8e30d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Volume 67d70a8b-31d3-4518-91f9-282305f5c110: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-67d70a8b-31d3-4518-91f9-282305f5c110'} >2018-08-22 03:18:52.732 39124 INFO cinder.volume.flows.manager.create_volume [req-49a9ec28-e566-41da-bb20-ae3545b8e30d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Volume volume-67d70a8b-31d3-4518-91f9-282305f5c110 (67d70a8b-31d3-4518-91f9-282305f5c110): created successfully >2018-08-22 03:18:52.736 39124 INFO cinder.volume.manager [req-49a9ec28-e566-41da-bb20-ae3545b8e30d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Created volume successfully. >2018-08-22 03:18:54.030 39124 INFO cinder.volume.flows.manager.create_volume [req-64d76c8c-f22c-4343-bbbf-c246c7335c06 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Volume fc012add-bb87-4380-811d-8cf4abb7e000: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fc012add-bb87-4380-811d-8cf4abb7e000'} >2018-08-22 03:18:56.442 39124 INFO cinder.volume.flows.manager.create_volume [req-64d76c8c-f22c-4343-bbbf-c246c7335c06 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Volume volume-fc012add-bb87-4380-811d-8cf4abb7e000 (fc012add-bb87-4380-811d-8cf4abb7e000): created successfully >2018-08-22 03:18:56.446 39124 INFO cinder.volume.manager [req-64d76c8c-f22c-4343-bbbf-c246c7335c06 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Created volume successfully. >2018-08-22 03:18:58.210 39124 INFO cinder.volume.flows.manager.create_volume [req-ea0b04b1-de4b-435b-bad5-4b82cf678a7d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Volume 372f0528-17c0-46fa-aea0-114be29503a3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-372f0528-17c0-46fa-aea0-114be29503a3'} >2018-08-22 03:18:58.840 39124 INFO cinder.volume.flows.manager.create_volume [req-ea0b04b1-de4b-435b-bad5-4b82cf678a7d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Volume volume-372f0528-17c0-46fa-aea0-114be29503a3 (372f0528-17c0-46fa-aea0-114be29503a3): created successfully >2018-08-22 03:18:58.843 39124 INFO cinder.volume.manager [req-ea0b04b1-de4b-435b-bad5-4b82cf678a7d 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Created volume successfully. >2018-08-22 03:19:00.288 39116 INFO cinder.volume.flows.manager.create_volume [req-032f9f24-04d3-4498-9fc1-97e7a1e23d10 48732022f3e24f31bc7fedc23ae9f308 920f6f725042460caf0bfaa1daacb998 - default default] Volume d8ef388e-dfb0-4453-9741-040c16d03ca5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d8ef388e-dfb0-4453-9741-040c16d03ca5'} >2018-08-22 03:19:00.831 39116 INFO cinder.volume.flows.manager.create_volume [req-032f9f24-04d3-4498-9fc1-97e7a1e23d10 48732022f3e24f31bc7fedc23ae9f308 920f6f725042460caf0bfaa1daacb998 - default default] Volume volume-d8ef388e-dfb0-4453-9741-040c16d03ca5 (d8ef388e-dfb0-4453-9741-040c16d03ca5): created successfully >2018-08-22 03:19:00.835 39116 INFO cinder.volume.manager [req-032f9f24-04d3-4498-9fc1-97e7a1e23d10 48732022f3e24f31bc7fedc23ae9f308 920f6f725042460caf0bfaa1daacb998 - default default] Created volume successfully. >2018-08-22 03:19:05.482 39116 INFO cinder.volume.manager [req-0a1137a3-7787-4988-b27a-3a959ae53486 48732022f3e24f31bc7fedc23ae9f308 920f6f725042460caf0bfaa1daacb998 - default default] Deleted volume successfully. >2018-08-22 03:19:05.505 39124 INFO cinder.volume.manager [req-288a75b2-370c-4641-bead-9576b0ac3382 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Deleted volume successfully. >2018-08-22 03:19:05.525 39124 INFO cinder.volume.manager [req-bb085709-d8e6-442f-b4e4-dadc8eed38d2 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Deleted volume successfully. >2018-08-22 03:19:05.729 39124 INFO cinder.volume.manager [req-817197cd-031b-496b-ab65-f7e7d715b726 594cbfa0b7f24f5681f625df38b621ab b8a00a7d02624a95b673d6b796e573a4 - default default] Deleted volume successfully. >2018-08-22 03:19:11.043 39124 INFO cinder.volume.flows.manager.create_volume [req-191b5783-f168-43df-8418-ea3d99bed647 8bd1e80d784c4c28b568c1d27c2a1819 2c56d5fc6b1441128fb4baf47c829435 - default default] Volume b2b5d133-7537-4f53-a61c-8de07728823a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b2b5d133-7537-4f53-a61c-8de07728823a'} >2018-08-22 03:19:11.661 39124 INFO cinder.volume.flows.manager.create_volume [req-191b5783-f168-43df-8418-ea3d99bed647 8bd1e80d784c4c28b568c1d27c2a1819 2c56d5fc6b1441128fb4baf47c829435 - default default] Volume volume-b2b5d133-7537-4f53-a61c-8de07728823a (b2b5d133-7537-4f53-a61c-8de07728823a): created successfully >2018-08-22 03:19:11.665 39124 INFO cinder.volume.manager [req-191b5783-f168-43df-8418-ea3d99bed647 8bd1e80d784c4c28b568c1d27c2a1819 2c56d5fc6b1441128fb4baf47c829435 - default default] Created volume successfully. >2018-08-22 03:19:15.818 39124 INFO cinder.volume.manager [req-18eb352a-1cc9-4bec-9c96-d30eaf474788 8bd1e80d784c4c28b568c1d27c2a1819 2c56d5fc6b1441128fb4baf47c829435 - default default] Deleted volume successfully. >2018-08-22 03:19:23.077 39124 INFO cinder.volume.flows.manager.create_volume [req-2d2fec12-24f5-42de-82e9-b3818f1065eb 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] Volume 5bcf88f6-dc10-4d5f-aa65-d6a8d703057c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5bcf88f6-dc10-4d5f-aa65-d6a8d703057c'} >2018-08-22 03:19:23.627 39124 INFO cinder.volume.flows.manager.create_volume [req-2d2fec12-24f5-42de-82e9-b3818f1065eb 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] Volume volume-5bcf88f6-dc10-4d5f-aa65-d6a8d703057c (5bcf88f6-dc10-4d5f-aa65-d6a8d703057c): created successfully >2018-08-22 03:19:23.631 39124 INFO cinder.volume.manager [req-2d2fec12-24f5-42de-82e9-b3818f1065eb 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] Created volume successfully. >2018-08-22 03:19:26.043 39124 INFO cinder.volume.manager [req-600efcb1-0e22-415c-b8fb-ae918cd1790f c0116a0aaede4b5e9bd14359d9fbacbe 9edbfc5354084c7d84e10410dca07b72 - default default] Transfer volume completed successfully. >2018-08-22 03:19:31.495 39124 INFO cinder.volume.manager [req-e6111033-6e5a-4ffe-9813-8f724db2c3f4 c7ce1dbc37144d92a3cd0d4d86691bd3 47ce4526cb2d48d8a67363a541f54267 - default default] Deleted volume successfully. >2018-08-22 03:19:32.139 39124 INFO cinder.volume.flows.manager.create_volume [req-df7b8e29-3ac5-4c39-bc5a-27fc22fc12ec 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] Volume 145eb2c5-6269-4b3b-8f8d-4f81e0f1c3ac: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-145eb2c5-6269-4b3b-8f8d-4f81e0f1c3ac'} >2018-08-22 03:19:32.750 39124 INFO cinder.volume.flows.manager.create_volume [req-df7b8e29-3ac5-4c39-bc5a-27fc22fc12ec 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] Volume volume-145eb2c5-6269-4b3b-8f8d-4f81e0f1c3ac (145eb2c5-6269-4b3b-8f8d-4f81e0f1c3ac): created successfully >2018-08-22 03:19:32.753 39124 INFO cinder.volume.manager [req-df7b8e29-3ac5-4c39-bc5a-27fc22fc12ec 0bb5cd581d4341b28c602f1f01bb17db 1509ce172abf4bbc95f81963e5021433 - default default] Created volume successfully. >2018-08-22 03:19:35.526 39124 INFO cinder.volume.manager [req-22d660f1-b906-494f-85d8-a8e44384e8d3 c7ce1dbc37144d92a3cd0d4d86691bd3 47ce4526cb2d48d8a67363a541f54267 - default default] Deleted volume successfully. >2018-08-22 03:19:41.408 39124 INFO cinder.volume.flows.manager.create_volume [req-1ee4c1a4-97e3-4e37-8a20-65b101f750e3 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Volume 57d35891-d4c9-4ed5-b91c-d8fe357eea86: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-57d35891-d4c9-4ed5-b91c-d8fe357eea86'} >2018-08-22 03:19:41.937 39124 INFO cinder.volume.flows.manager.create_volume [req-1ee4c1a4-97e3-4e37-8a20-65b101f750e3 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Volume volume-57d35891-d4c9-4ed5-b91c-d8fe357eea86 (57d35891-d4c9-4ed5-b91c-d8fe357eea86): created successfully >2018-08-22 03:19:41.941 39124 INFO cinder.volume.manager [req-1ee4c1a4-97e3-4e37-8a20-65b101f750e3 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Created volume successfully. >2018-08-22 03:19:49.770 39124 INFO cinder.volume.manager [req-7c347806-62e7-4571-989c-5e090362a5ff ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Attaching volume 57d35891-d4c9-4ed5-b91c-d8fe357eea86 to instance 18ab2f2c-bc27-4236-89fa-6b5ac74a1891 at mountpoint /dev/vdb on host None. >2018-08-22 03:19:49.858 39124 INFO cinder.volume.manager [req-7c347806-62e7-4571-989c-5e090362a5ff ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Attach volume completed successfully. >2018-08-22 03:19:50.112 39124 INFO cinder.volume.manager [req-8ff37e00-0231-40bd-9386-2727cdeaea76 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Detaching volume 57d35891-d4c9-4ed5-b91c-d8fe357eea86 from instance 18ab2f2c-bc27-4236-89fa-6b5ac74a1891. >2018-08-22 03:19:50.212 39124 INFO cinder.volume.manager [req-8ff37e00-0231-40bd-9386-2727cdeaea76 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Detach volume completed successfully. >2018-08-22 03:19:56.458 39124 INFO cinder.volume.manager [req-b6bcdaa3-6301-43a9-ab26-1072e56f760e ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Attaching volume 57d35891-d4c9-4ed5-b91c-d8fe357eea86 to instance 6c4a7842-dbb9-4f04-84ed-25ce1d060e90 at mountpoint /dev/vdb on host None. >2018-08-22 03:19:56.535 39124 INFO cinder.volume.manager [req-b6bcdaa3-6301-43a9-ab26-1072e56f760e ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Attach volume completed successfully. >2018-08-22 03:19:57.275 39124 INFO cinder.volume.manager [req-3f54fe56-677b-4696-a7aa-8f63006a5962 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Detaching volume 57d35891-d4c9-4ed5-b91c-d8fe357eea86 from instance 6c4a7842-dbb9-4f04-84ed-25ce1d060e90. >2018-08-22 03:19:57.378 39124 INFO cinder.volume.manager [req-3f54fe56-677b-4696-a7aa-8f63006a5962 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Detach volume completed successfully. >2018-08-22 03:20:09.431 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-13901dbc-5c32-4fb3-846e-d1d64c49bf50 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] initialize_connection: Did not find a preferred node for volume volume-57d35891-d4c9-4ed5-b91c-d8fe357eea86. >2018-08-22 03:20:10.066 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 03:20:27.878 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-13901dbc-5c32-4fb3-846e-d1d64c49bf50 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:20:28.412 39124 INFO cinder.volume.manager [req-13901dbc-5c32-4fb3-846e-d1d64c49bf50 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Copy volume to image completed successfully. >2018-08-22 03:20:32.602 39124 INFO cinder.volume.manager [req-20e78968-1c1c-4f6c-9aac-5cf2c6b1e3e8 ab3821b51a7b4589ab3b863a5b79debb 42c9d799f03b484b9ff64cb7853b2e11 - default default] Deleted volume successfully. >2018-08-22 03:20:39.335 39124 INFO cinder.volume.flows.manager.create_volume [req-2f915e3c-edb8-4d24-86e3-5bb7fbfc1604 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Volume aaf2bd18-6e45-4805-bc77-7fe8f5341c2c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-aaf2bd18-6e45-4805-bc77-7fe8f5341c2c'} >2018-08-22 03:20:40.297 39124 INFO cinder.volume.flows.manager.create_volume [req-2f915e3c-edb8-4d24-86e3-5bb7fbfc1604 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Volume volume-aaf2bd18-6e45-4805-bc77-7fe8f5341c2c (aaf2bd18-6e45-4805-bc77-7fe8f5341c2c): created successfully >2018-08-22 03:20:40.301 39124 INFO cinder.volume.manager [req-2f915e3c-edb8-4d24-86e3-5bb7fbfc1604 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Created volume successfully. >2018-08-22 03:21:04.278 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-74a633e6-245e-4883-9f49-9aec3f1bbc97 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] initialize_connection: Did not find a preferred node for volume volume-aaf2bd18-6e45-4805-bc77-7fe8f5341c2c. >2018-08-22 03:21:04.753 39124 INFO cinder.volume.manager [req-74a633e6-245e-4883-9f49-9aec3f1bbc97 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Initialize volume connection completed successfully. >2018-08-22 03:21:09.453 39124 INFO cinder.volume.manager [req-e40ee357-f331-4e1a-903c-351ad8dc2fcb 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Attaching volume aaf2bd18-6e45-4805-bc77-7fe8f5341c2c to instance 30c2dc05-978c-4787-aad6-c7279fa1719a at mountpoint /dev/vdb on host None. >2018-08-22 03:21:09.524 39124 INFO cinder.volume.manager [req-e40ee357-f331-4e1a-903c-351ad8dc2fcb 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Attach volume completed successfully. >2018-08-22 03:21:10.541 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e925f0d9-e967-47af-b5c9-d688aa38078d 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Unable to get remote copy information for volume volume-aaf2bd18-6e45-4805-bc77-7fe8f5341c2c >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova [req-e925f0d9-e967-47af-b5c9-d688aa38078d 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Failed to notify nova on events: [{'tag': 'aaf2bd18-6e45-4805-bc77-7fe8f5341c2c', 'name': 'volume-extended', 'server_uuid': '30c2dc05-978c-4787-aad6-c7279fa1719a'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-a4108fb6-56d3-40cd-9ca2-a355f1bc35da) >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova return_raw=True) >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-a4108fb6-56d3-40cd-9ca2-a355f1bc35da) >2018-08-22 03:21:11.456 39124 ERROR cinder.compute.nova >2018-08-22 03:21:11.479 39124 INFO cinder.volume.manager [req-e925f0d9-e967-47af-b5c9-d688aa38078d 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Extend volume completed successfully. >2018-08-22 03:23:56.363 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f59f3e54-6173-4cc3-b972-03109c3be48d 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:23:56.642 39124 INFO cinder.volume.manager [req-f59f3e54-6173-4cc3-b972-03109c3be48d 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Terminate volume connection completed successfully. >2018-08-22 03:23:56.801 39124 INFO cinder.volume.manager [req-aa0209af-8339-4a00-9296-68a60abf5d6e 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Detaching volume aaf2bd18-6e45-4805-bc77-7fe8f5341c2c from instance 30c2dc05-978c-4787-aad6-c7279fa1719a. >2018-08-22 03:23:56.887 39124 INFO cinder.volume.manager [req-aa0209af-8339-4a00-9296-68a60abf5d6e 91a3450ce81f4cab9b857a17d502e10c 154bf7b7ee93459d9d59906028556434 - default default] Detach volume completed successfully. >2018-08-22 03:39:08.143 39124 INFO cinder.volume.manager [req-5bf78172-af78-4e91-b4cc-428e7de0f0a4 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-1506994536: creating >2018-08-22 03:39:08.169 39124 INFO cinder.volume.manager [req-5bf78172-af78-4e91-b4cc-428e7de0f0a4 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-1506994536: created successfully >2018-08-22 03:39:08.192 39124 INFO cinder.volume.manager [req-5bf78172-af78-4e91-b4cc-428e7de0f0a4 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Create group completed successfully. >2018-08-22 03:39:08.762 39124 INFO cinder.volume.flows.manager.create_volume [req-171b539c-249d-44e0-8d72-4b9ccd910820 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume ff95e71a-faba-416d-a16a-70a161120ece: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ff95e71a-faba-416d-a16a-70a161120ece'} >2018-08-22 03:39:09.313 39124 INFO cinder.volume.flows.manager.create_volume [req-171b539c-249d-44e0-8d72-4b9ccd910820 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume volume-ff95e71a-faba-416d-a16a-70a161120ece (ff95e71a-faba-416d-a16a-70a161120ece): created successfully >2018-08-22 03:39:09.317 39124 INFO cinder.volume.manager [req-171b539c-249d-44e0-8d72-4b9ccd910820 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Created volume successfully. >2018-08-22 03:39:10.663 39124 INFO cinder.volume.manager [req-5f787104-10d4-4c56-8632-be67fbc9e225 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] GroupSnapshot 188b7dfd-0986-4f66-9155-6a735b6ca949: creating. >2018-08-22 03:39:12.944 39124 INFO cinder.volume.manager [req-5f787104-10d4-4c56-8632-be67fbc9e225 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] group_snapshot 188b7dfd-0986-4f66-9155-6a735b6ca949: created successfully >2018-08-22 03:39:13.554 39124 INFO cinder.volume.manager [req-236b5591-60ce-4d42-a14d-2ba31c9bbc5c 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] group_snapshot 188b7dfd-0986-4f66-9155-6a735b6ca949: deleting >2018-08-22 03:39:15.935 39124 WARNING cinder.quota [req-236b5591-60ce-4d42-a14d-2ba31c9bbc5c 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-31455452 is set by the default quota flag: quota_snapshots_tempest-volume-type-31455452, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:15.967 39124 INFO cinder.volume.manager [req-236b5591-60ce-4d42-a14d-2ba31c9bbc5c 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] group_snapshot 188b7dfd-0986-4f66-9155-6a735b6ca949: deleted successfully >2018-08-22 03:39:18.005 39124 WARNING cinder.quota [req-e6c70032-1515-4c39-95f3-6d0efb55a3ab 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-31455452 is set by the default quota flag: quota_volumes_tempest-volume-type-31455452, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:18.007 39124 WARNING cinder.quota [req-e6c70032-1515-4c39-95f3-6d0efb55a3ab 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-31455452 is set by the default quota flag: quota_gigabytes_tempest-volume-type-31455452, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:18.462 39124 INFO cinder.volume.manager [req-e6c70032-1515-4c39-95f3-6d0efb55a3ab 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Delete group completed successfully. >2018-08-22 03:39:19.387 39124 INFO cinder.volume.manager [req-54128723-bdfd-4e43-818b-fcaac2e69a03 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-1384899550: creating >2018-08-22 03:39:19.408 39124 INFO cinder.volume.manager [req-54128723-bdfd-4e43-818b-fcaac2e69a03 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-1384899550: created successfully >2018-08-22 03:39:19.427 39124 INFO cinder.volume.manager [req-54128723-bdfd-4e43-818b-fcaac2e69a03 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Create group completed successfully. >2018-08-22 03:39:19.789 39124 INFO cinder.volume.flows.manager.create_volume [req-6ed9716d-5051-4ac2-b29b-607f4c049eb0 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume cf214ad5-1486-4f1b-99e2-50e79e538970: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cf214ad5-1486-4f1b-99e2-50e79e538970'} >2018-08-22 03:39:20.675 39124 INFO cinder.volume.flows.manager.create_volume [req-6ed9716d-5051-4ac2-b29b-607f4c049eb0 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume volume-cf214ad5-1486-4f1b-99e2-50e79e538970 (cf214ad5-1486-4f1b-99e2-50e79e538970): created successfully >2018-08-22 03:39:20.678 39124 INFO cinder.volume.manager [req-6ed9716d-5051-4ac2-b29b-607f4c049eb0 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Created volume successfully. >2018-08-22 03:39:22.635 39124 WARNING cinder.quota [req-9357bd04-32fd-483a-bafe-df56321b3233 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-580385437 is set by the default quota flag: quota_volumes_tempest-volume-type-580385437, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:22.636 39124 WARNING cinder.quota [req-9357bd04-32fd-483a-bafe-df56321b3233 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-580385437 is set by the default quota flag: quota_gigabytes_tempest-volume-type-580385437, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:23.107 39124 INFO cinder.volume.manager [req-9357bd04-32fd-483a-bafe-df56321b3233 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Delete group completed successfully. >2018-08-22 03:39:23.765 39124 INFO cinder.volume.manager [req-4c405ccd-e9cb-4662-87df-81c675752ff1 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-1214679774: creating >2018-08-22 03:39:23.789 39124 INFO cinder.volume.manager [req-4c405ccd-e9cb-4662-87df-81c675752ff1 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-1214679774: created successfully >2018-08-22 03:39:23.811 39124 INFO cinder.volume.manager [req-4c405ccd-e9cb-4662-87df-81c675752ff1 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Create group completed successfully. >2018-08-22 03:39:24.372 39124 INFO cinder.volume.flows.manager.create_volume [req-cb9aef59-3678-4d50-b7ae-7aa03832e980 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume 0ad6963e-258a-41aa-95dc-a18b7d1c5c37: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0ad6963e-258a-41aa-95dc-a18b7d1c5c37'} >2018-08-22 03:39:25.244 39124 INFO cinder.volume.flows.manager.create_volume [req-cb9aef59-3678-4d50-b7ae-7aa03832e980 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume volume-0ad6963e-258a-41aa-95dc-a18b7d1c5c37 (0ad6963e-258a-41aa-95dc-a18b7d1c5c37): created successfully >2018-08-22 03:39:25.247 39124 INFO cinder.volume.manager [req-cb9aef59-3678-4d50-b7ae-7aa03832e980 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Created volume successfully. >2018-08-22 03:39:25.602 39124 INFO cinder.volume.manager [req-4787a0de-0d00-4fc5-97c1-5438e96d041b 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] GroupSnapshot 37aa48d5-fbc8-4b6e-957c-55e94f433d0d: creating. >2018-08-22 03:39:29.842 39124 INFO cinder.volume.manager [req-4787a0de-0d00-4fc5-97c1-5438e96d041b 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] group_snapshot 37aa48d5-fbc8-4b6e-957c-55e94f433d0d: created successfully >2018-08-22 03:39:37.038 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-06f16a33-9c9d-4722-8d6d-e81c9e96b2a6 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Updating status for CG: 623eb305-09bd-499c-94f3-09d3665f3f8b. >2018-08-22 03:39:37.145 39124 INFO cinder.volume.manager [req-06f16a33-9c9d-4722-8d6d-e81c9e96b2a6 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Create group from source-snapshot-37aa48d5-fbc8-4b6e-957c-55e94f433d0d completed successfully. >2018-08-22 03:39:41.361 39124 WARNING cinder.quota [req-aebaf38d-71f7-40fe-892f-ecb7bec270ff 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1993611967 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1993611967, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:41.362 39124 WARNING cinder.quota [req-aebaf38d-71f7-40fe-892f-ecb7bec270ff 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1993611967 is set by the default quota flag: quota_volumes_tempest-volume-type-1993611967, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:41.645 39124 INFO cinder.volume.manager [req-aebaf38d-71f7-40fe-892f-ecb7bec270ff 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Delete group completed successfully. >2018-08-22 03:39:41.940 39124 INFO cinder.volume.manager [req-1c425928-8118-459a-9980-a40edc59809c 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] group_snapshot 37aa48d5-fbc8-4b6e-957c-55e94f433d0d: deleting >2018-08-22 03:39:44.037 39124 WARNING cinder.quota [req-1c425928-8118-459a-9980-a40edc59809c 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-1993611967 is set by the default quota flag: quota_snapshots_tempest-volume-type-1993611967, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:44.073 39124 INFO cinder.volume.manager [req-1c425928-8118-459a-9980-a40edc59809c 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] group_snapshot 37aa48d5-fbc8-4b6e-957c-55e94f433d0d: deleted successfully >2018-08-22 03:39:45.862 39124 INFO cinder.volume.manager [req-0e138295-dd35-474c-bc64-0a2876fa1da6 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Delete group completed successfully. >2018-08-22 03:39:46.074 39124 INFO cinder.volume.manager [req-0fed0396-1e8c-4910-baa6-41591a8f1f7a 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-147602594: creating >2018-08-22 03:39:46.096 39124 INFO cinder.volume.manager [req-0fed0396-1e8c-4910-baa6-41591a8f1f7a 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Group tempest-CG-147602594: created successfully >2018-08-22 03:39:46.119 39124 INFO cinder.volume.manager [req-0fed0396-1e8c-4910-baa6-41591a8f1f7a 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Create group completed successfully. >2018-08-22 03:39:46.472 39124 INFO cinder.volume.flows.manager.create_volume [req-00bf613f-f4fa-434c-9876-15b4297b8087 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume 2bac30cf-74dd-42a9-a67e-2d7e680e6bca: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2bac30cf-74dd-42a9-a67e-2d7e680e6bca'} >2018-08-22 03:39:47.040 39124 INFO cinder.volume.flows.manager.create_volume [req-00bf613f-f4fa-434c-9876-15b4297b8087 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Volume volume-2bac30cf-74dd-42a9-a67e-2d7e680e6bca (2bac30cf-74dd-42a9-a67e-2d7e680e6bca): created successfully >2018-08-22 03:39:47.044 39124 INFO cinder.volume.manager [req-00bf613f-f4fa-434c-9876-15b4297b8087 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Created volume successfully. >2018-08-22 03:39:50.094 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fcfe44aa-90c4-4ac7-b9cf-0050058a68b4 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Updating status for CG: e355a81b-4f9f-4d4e-9403-3b71270ccbbd. >2018-08-22 03:39:50.187 39124 INFO cinder.volume.manager [req-fcfe44aa-90c4-4ac7-b9cf-0050058a68b4 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Create group from source-group-832b595e-ac25-4eb9-a032-dee84169ceaa completed successfully. >2018-08-22 03:39:56.850 39124 WARNING cinder.quota [req-437936bf-9224-44a5-aab7-68eb859bcd62 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1756931196 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1756931196, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:56.851 39124 WARNING cinder.quota [req-437936bf-9224-44a5-aab7-68eb859bcd62 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1756931196 is set by the default quota flag: quota_volumes_tempest-volume-type-1756931196, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:39:57.146 39124 INFO cinder.volume.manager [req-437936bf-9224-44a5-aab7-68eb859bcd62 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Delete group completed successfully. >2018-08-22 03:39:58.879 39124 INFO cinder.volume.manager [req-f5f3bd35-afa3-4396-85dc-fe26ed477b64 2e17e581ea514cbb836b33efa5628200 de187c5a38dd4870a54d7e2d4425685a - default default] Delete group completed successfully. >2018-08-22 03:41:08.609 39124 INFO cinder.volume.flows.manager.create_volume [req-0f0d0aff-1294-443d-aaf8-998bcbb4ec65 93160c876e2b45db96cd2efc0ecb6a30 2930afa0cf8743e5b27514fe17d98d45 - default default] Volume 819f4791-eb64-4632-9055-27a6381478f1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-819f4791-eb64-4632-9055-27a6381478f1'} >2018-08-22 03:41:09.420 39124 INFO cinder.volume.flows.manager.create_volume [req-0f0d0aff-1294-443d-aaf8-998bcbb4ec65 93160c876e2b45db96cd2efc0ecb6a30 2930afa0cf8743e5b27514fe17d98d45 - default default] Volume volume-819f4791-eb64-4632-9055-27a6381478f1 (819f4791-eb64-4632-9055-27a6381478f1): created successfully >2018-08-22 03:41:09.424 39124 INFO cinder.volume.manager [req-0f0d0aff-1294-443d-aaf8-998bcbb4ec65 93160c876e2b45db96cd2efc0ecb6a30 2930afa0cf8743e5b27514fe17d98d45 - default default] Created volume successfully. >2018-08-22 03:41:11.789 39124 INFO cinder.volume.manager [req-97604790-92b5-4d49-ad03-608e56b392a4 93160c876e2b45db96cd2efc0ecb6a30 2930afa0cf8743e5b27514fe17d98d45 - default default] Deleted volume successfully. >2018-08-22 03:41:21.127 39124 INFO cinder.volume.flows.manager.create_volume [req-0b05c6a6-779c-432a-88de-64627b97dba8 c7426498a6fc4a99a0247222116695c7 4551bf87de28426abf81d73bf0f7ab8c - default default] Volume 182b126a-dd63-4428-a4dc-728d8dce3fc7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-182b126a-dd63-4428-a4dc-728d8dce3fc7'} >2018-08-22 03:41:22.047 39124 INFO cinder.volume.flows.manager.create_volume [req-0b05c6a6-779c-432a-88de-64627b97dba8 c7426498a6fc4a99a0247222116695c7 4551bf87de28426abf81d73bf0f7ab8c - default default] Volume volume-182b126a-dd63-4428-a4dc-728d8dce3fc7 (182b126a-dd63-4428-a4dc-728d8dce3fc7): created successfully >2018-08-22 03:41:22.051 39124 INFO cinder.volume.manager [req-0b05c6a6-779c-432a-88de-64627b97dba8 c7426498a6fc4a99a0247222116695c7 4551bf87de28426abf81d73bf0f7ab8c - default default] Created volume successfully. >2018-08-22 03:41:25.310 39124 WARNING cinder.quota [req-34bd5241-b180-4f08-b3c6-fe6266d701a3 c7426498a6fc4a99a0247222116695c7 4551bf87de28426abf81d73bf0f7ab8c - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1754549727 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1754549727, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:41:25.312 39124 WARNING cinder.quota [req-34bd5241-b180-4f08-b3c6-fe6266d701a3 c7426498a6fc4a99a0247222116695c7 4551bf87de28426abf81d73bf0f7ab8c - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1754549727 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1754549727, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:41:25.579 39124 INFO cinder.volume.manager [req-34bd5241-b180-4f08-b3c6-fe6266d701a3 c7426498a6fc4a99a0247222116695c7 4551bf87de28426abf81d73bf0f7ab8c - default default] Deleted volume successfully. >2018-08-22 03:41:35.215 39124 INFO cinder.volume.flows.manager.create_volume [req-f7fba6d8-b5c8-4850-b914-abaa5639333c 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] Volume 29d6afa5-a7a0-4959-8c8e-b84978421ef6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-29d6afa5-a7a0-4959-8c8e-b84978421ef6'} >2018-08-22 03:41:35.773 39124 INFO cinder.volume.flows.manager.create_volume [req-f7fba6d8-b5c8-4850-b914-abaa5639333c 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] Volume volume-29d6afa5-a7a0-4959-8c8e-b84978421ef6 (29d6afa5-a7a0-4959-8c8e-b84978421ef6): created successfully >2018-08-22 03:41:35.777 39124 INFO cinder.volume.manager [req-f7fba6d8-b5c8-4850-b914-abaa5639333c 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] Created volume successfully. >2018-08-22 03:41:37.357 39124 INFO cinder.volume.manager [req-6eec8215-c818-4f7b-8bd6-bc074a34ebb7 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] Retype volume completed successfully. >2018-08-22 03:41:40.862 39124 WARNING cinder.quota [req-3aa8b8c3-5230-46d7-bb23-e3f8f3604abe 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1873302887 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1873302887, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:41:40.863 39124 WARNING cinder.quota [req-3aa8b8c3-5230-46d7-bb23-e3f8f3604abe 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1873302887 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1873302887, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:41:41.120 39124 INFO cinder.volume.manager [req-3aa8b8c3-5230-46d7-bb23-e3f8f3604abe 6cf1d4bc705a4895b37017520c23343c 33e0cde7b138475ebf8dc2262a934ca3 - default default] Deleted volume successfully. >2018-08-22 03:42:19.012 39124 INFO cinder.volume.flows.manager.create_volume [req-7086d194-87a5-49b6-a8fc-13a871097961 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume 14f69973-ee9c-4d57-8110-3c139fe7bce8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-14f69973-ee9c-4d57-8110-3c139fe7bce8'} >2018-08-22 03:42:19.866 39124 INFO cinder.volume.flows.manager.create_volume [req-7086d194-87a5-49b6-a8fc-13a871097961 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume volume-14f69973-ee9c-4d57-8110-3c139fe7bce8 (14f69973-ee9c-4d57-8110-3c139fe7bce8): created successfully >2018-08-22 03:42:19.870 39124 INFO cinder.volume.manager [req-7086d194-87a5-49b6-a8fc-13a871097961 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Created volume successfully. >2018-08-22 03:42:20.641 39124 INFO cinder.volume.manager [req-055f9d2e-f0a0-4faf-b78b-ca2a63534dac c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Attaching volume 14f69973-ee9c-4d57-8110-3c139fe7bce8 to instance b8a45e97-04f6-40e7-9a43-1ce4caba9b5e at mountpoint /dev/vdb on host None. >2018-08-22 03:42:20.709 39124 INFO cinder.volume.manager [req-055f9d2e-f0a0-4faf-b78b-ca2a63534dac c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Attach volume completed successfully. >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager [req-e41d47bb-aae5-401d-a9e6-bc8e82cec34f a8df86e35bde4b9da8a966ce0f6ee8e0 f8249fce6af840408ffe78b9be1630a0 - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager force=force) >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-22 03:42:22.002 39124 ERROR cinder.volume.manager >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server [req-e41d47bb-aae5-401d-a9e6-bc8e82cec34f a8df86e35bde4b9da8a966ce0f6ee8e0 f8249fce6af840408ffe78b9be1630a0 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 03:42:22.004 39124 ERROR oslo_messaging.rpc.server >2018-08-22 03:42:22.132 39124 INFO cinder.volume.manager [req-643a4f6a-8110-47e7-8d72-772f65c8b3c2 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Detaching volume 14f69973-ee9c-4d57-8110-3c139fe7bce8 from instance b8a45e97-04f6-40e7-9a43-1ce4caba9b5e. >2018-08-22 03:42:22.230 39124 INFO cinder.volume.manager [req-643a4f6a-8110-47e7-8d72-772f65c8b3c2 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Detach volume completed successfully. >2018-08-22 03:42:24.289 39124 INFO cinder.volume.flows.manager.create_volume [req-27c96d06-a4ef-4b25-9d6d-27bdc84502a0 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume 00cd5275-6f21-4bb3-b563-4e8a766e3228: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-00cd5275-6f21-4bb3-b563-4e8a766e3228'} >2018-08-22 03:42:25.338 39124 INFO cinder.volume.flows.manager.create_volume [req-27c96d06-a4ef-4b25-9d6d-27bdc84502a0 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume volume-00cd5275-6f21-4bb3-b563-4e8a766e3228 (00cd5275-6f21-4bb3-b563-4e8a766e3228): created successfully >2018-08-22 03:42:25.342 39124 INFO cinder.volume.manager [req-27c96d06-a4ef-4b25-9d6d-27bdc84502a0 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Created volume successfully. >2018-08-22 03:42:31.909 39124 INFO cinder.volume.manager [req-8a88299b-18de-4525-ad57-24b506f40925 a8df86e35bde4b9da8a966ce0f6ee8e0 f8249fce6af840408ffe78b9be1630a0 - default default] Deleted volume successfully. >2018-08-22 03:42:32.522 39124 INFO cinder.volume.flows.manager.create_volume [req-182eb442-635e-464d-9307-0c3d8e4eb7e2 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume 0237e184-7613-48bc-a531-ff46fc6eb122: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0237e184-7613-48bc-a531-ff46fc6eb122'} >2018-08-22 03:42:33.578 39124 INFO cinder.volume.flows.manager.create_volume [req-182eb442-635e-464d-9307-0c3d8e4eb7e2 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume volume-0237e184-7613-48bc-a531-ff46fc6eb122 (0237e184-7613-48bc-a531-ff46fc6eb122): created successfully >2018-08-22 03:42:33.582 39124 INFO cinder.volume.manager [req-182eb442-635e-464d-9307-0c3d8e4eb7e2 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Created volume successfully. >2018-08-22 03:42:37.195 39124 INFO cinder.volume.manager [req-37fdc77d-1c03-4057-bb3a-640cf9bc9c93 a8df86e35bde4b9da8a966ce0f6ee8e0 f8249fce6af840408ffe78b9be1630a0 - default default] Deleted volume successfully. >2018-08-22 03:42:37.737 39124 INFO cinder.volume.flows.manager.create_volume [req-d6c545f0-55dc-4968-b317-46e5ff4524d9 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume b0608c24-c199-48a1-bac7-639c8b3c05d3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b0608c24-c199-48a1-bac7-639c8b3c05d3'} >2018-08-22 03:42:38.840 39124 INFO cinder.volume.flows.manager.create_volume [req-d6c545f0-55dc-4968-b317-46e5ff4524d9 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume volume-b0608c24-c199-48a1-bac7-639c8b3c05d3 (b0608c24-c199-48a1-bac7-639c8b3c05d3): created successfully >2018-08-22 03:42:38.844 39124 INFO cinder.volume.manager [req-d6c545f0-55dc-4968-b317-46e5ff4524d9 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Created volume successfully. >2018-08-22 03:42:42.798 39124 INFO cinder.volume.manager [req-0549218d-2334-486e-ad54-75f918ae81be a8df86e35bde4b9da8a966ce0f6ee8e0 f8249fce6af840408ffe78b9be1630a0 - default default] Deleted volume successfully. >2018-08-22 03:42:43.360 39124 INFO cinder.volume.flows.manager.create_volume [req-97934898-dba8-45e6-8029-944dcd6d40ca c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume 93c85fc1-1806-4f54-ae52-e6966c74ce0f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-93c85fc1-1806-4f54-ae52-e6966c74ce0f'} >2018-08-22 03:42:44.424 39124 INFO cinder.volume.flows.manager.create_volume [req-97934898-dba8-45e6-8029-944dcd6d40ca c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume volume-93c85fc1-1806-4f54-ae52-e6966c74ce0f (93c85fc1-1806-4f54-ae52-e6966c74ce0f): created successfully >2018-08-22 03:42:44.428 39124 INFO cinder.volume.manager [req-97934898-dba8-45e6-8029-944dcd6d40ca c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Created volume successfully. >2018-08-22 03:42:50.291 39124 INFO cinder.volume.manager [req-57cda838-6648-40bb-91f6-3f494c08357b a8df86e35bde4b9da8a966ce0f6ee8e0 f8249fce6af840408ffe78b9be1630a0 - default default] Deleted volume successfully. >2018-08-22 03:42:50.470 39124 INFO cinder.volume.flows.manager.create_volume [req-579d95ad-0171-450d-a9a0-503a7bcc74a8 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume f5f84aa8-e466-436e-9503-60d75f2ce4b3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f5f84aa8-e466-436e-9503-60d75f2ce4b3'} >2018-08-22 03:42:51.398 39124 INFO cinder.volume.flows.manager.create_volume [req-579d95ad-0171-450d-a9a0-503a7bcc74a8 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Volume volume-f5f84aa8-e466-436e-9503-60d75f2ce4b3 (f5f84aa8-e466-436e-9503-60d75f2ce4b3): created successfully >2018-08-22 03:42:51.402 39124 INFO cinder.volume.manager [req-579d95ad-0171-450d-a9a0-503a7bcc74a8 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Created volume successfully. >2018-08-22 03:42:55.009 39124 INFO cinder.volume.manager [req-5f882fdd-d442-4598-964f-66ac3347d491 c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Deleted volume successfully. >2018-08-22 03:42:55.434 39124 INFO cinder.volume.manager [req-9241c681-349e-462e-9fc2-a2d1e83061bf c6ac9b81b61247399db3410bfd468765 c56c4b2b9c5c45f2a17d50fdc355d295 - default default] Deleted volume successfully. >2018-08-22 03:43:03.010 39124 INFO cinder.volume.flows.manager.create_volume [req-52632020-6452-4351-b7da-aff611c11fc5 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Volume 241a52db-2966-4e63-8295-19e558afbd7b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-241a52db-2966-4e63-8295-19e558afbd7b'} >2018-08-22 03:43:03.853 39124 INFO cinder.volume.flows.manager.create_volume [req-52632020-6452-4351-b7da-aff611c11fc5 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Volume volume-241a52db-2966-4e63-8295-19e558afbd7b (241a52db-2966-4e63-8295-19e558afbd7b): created successfully >2018-08-22 03:43:03.858 39124 INFO cinder.volume.manager [req-52632020-6452-4351-b7da-aff611c11fc5 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Created volume successfully. >2018-08-22 03:43:05.100 39124 INFO cinder.volume.flows.manager.create_volume [req-5db4f1d0-93b1-420d-9191-dae308ae4702 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Volume 15187e88-4501-458c-ad65-250abc560ca4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-15187e88-4501-458c-ad65-250abc560ca4'} >2018-08-22 03:43:06.169 39124 INFO cinder.volume.flows.manager.create_volume [req-5db4f1d0-93b1-420d-9191-dae308ae4702 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Volume volume-15187e88-4501-458c-ad65-250abc560ca4 (15187e88-4501-458c-ad65-250abc560ca4): created successfully >2018-08-22 03:43:06.173 39124 INFO cinder.volume.manager [req-5db4f1d0-93b1-420d-9191-dae308ae4702 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Created volume successfully. >2018-08-22 03:43:07.784 39116 INFO cinder.volume.flows.manager.create_volume [req-cb65d07d-564f-4bbc-80ee-6163897c3c1e 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Volume 81bb95c3-25db-43a2-ac8a-7b2177cb9e67: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-81bb95c3-25db-43a2-ac8a-7b2177cb9e67'} >2018-08-22 03:43:08.829 39116 INFO cinder.volume.flows.manager.create_volume [req-cb65d07d-564f-4bbc-80ee-6163897c3c1e 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Volume volume-81bb95c3-25db-43a2-ac8a-7b2177cb9e67 (81bb95c3-25db-43a2-ac8a-7b2177cb9e67): created successfully >2018-08-22 03:43:08.833 39116 INFO cinder.volume.manager [req-cb65d07d-564f-4bbc-80ee-6163897c3c1e 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Created volume successfully. >2018-08-22 03:43:09.607 39124 INFO cinder.volume.flows.manager.create_volume [req-ba9088e3-9e94-4093-99cb-9f75738dbb86 bc5e2c633d2e40a8ab4e034989822a22 7b6a66780cef45f9b6727db6fb0fcb86 - default default] Volume f891e76d-fde3-4699-a8c4-14f36c787801: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f891e76d-fde3-4699-a8c4-14f36c787801'} >2018-08-22 03:43:10.775 39124 INFO cinder.volume.flows.manager.create_volume [req-ba9088e3-9e94-4093-99cb-9f75738dbb86 bc5e2c633d2e40a8ab4e034989822a22 7b6a66780cef45f9b6727db6fb0fcb86 - default default] Volume volume-f891e76d-fde3-4699-a8c4-14f36c787801 (f891e76d-fde3-4699-a8c4-14f36c787801): created successfully >2018-08-22 03:43:10.779 39124 INFO cinder.volume.manager [req-ba9088e3-9e94-4093-99cb-9f75738dbb86 bc5e2c633d2e40a8ab4e034989822a22 7b6a66780cef45f9b6727db6fb0fcb86 - default default] Created volume successfully. >2018-08-22 03:43:15.795 39124 INFO cinder.volume.manager [req-c693135d-50c8-484a-9fc9-6e3b95b69305 bc5e2c633d2e40a8ab4e034989822a22 7b6a66780cef45f9b6727db6fb0fcb86 - default default] Deleted volume successfully. >2018-08-22 03:43:15.896 39116 INFO cinder.volume.manager [req-dbeebfcd-358f-47bb-8327-f78dbea82072 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Deleted volume successfully. >2018-08-22 03:43:16.168 39124 INFO cinder.volume.manager [req-8fdee96c-426f-433a-9c9f-82b39ff7f545 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Deleted volume successfully. >2018-08-22 03:43:16.405 39124 INFO cinder.volume.manager [req-105d1249-5b5b-44f0-b90a-d330489b04f9 0a179e5cd14b4168875f776665bc1344 98f2636c3bb54ec29af90da67ddd0a4a - default default] Deleted volume successfully. >2018-08-22 03:43:21.140 39124 INFO cinder.volume.flows.manager.create_volume [req-d1d478ce-d29e-4fcc-a2ae-3599a25a4268 4f45160dabcf4915adf452fe60511450 a2add2ba91fb40599b6976b809fb23b9 - default default] Volume 0ac4b324-2f19-4c68-ad29-e6c2f7636bd4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0ac4b324-2f19-4c68-ad29-e6c2f7636bd4'} >2018-08-22 03:43:22.124 39124 INFO cinder.volume.flows.manager.create_volume [req-d1d478ce-d29e-4fcc-a2ae-3599a25a4268 4f45160dabcf4915adf452fe60511450 a2add2ba91fb40599b6976b809fb23b9 - default default] Volume volume-0ac4b324-2f19-4c68-ad29-e6c2f7636bd4 (0ac4b324-2f19-4c68-ad29-e6c2f7636bd4): created successfully >2018-08-22 03:43:22.127 39124 INFO cinder.volume.manager [req-d1d478ce-d29e-4fcc-a2ae-3599a25a4268 4f45160dabcf4915adf452fe60511450 a2add2ba91fb40599b6976b809fb23b9 - default default] Created volume successfully. >2018-08-22 03:43:26.398 39124 INFO cinder.volume.manager [req-c3390f45-e1f9-41d8-a8cc-e928c12156b9 4f45160dabcf4915adf452fe60511450 a2add2ba91fb40599b6976b809fb23b9 - default default] Deleted volume successfully. >2018-08-22 03:43:33.524 39124 INFO cinder.volume.flows.manager.create_volume [req-e65639c7-d6f8-4e3f-83c9-d781178fd56c baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] Volume 270a0ace-7f68-42f8-b371-32e7cc7a8ffc: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-270a0ace-7f68-42f8-b371-32e7cc7a8ffc'} >2018-08-22 03:43:34.186 39124 INFO cinder.volume.flows.manager.create_volume [req-e65639c7-d6f8-4e3f-83c9-d781178fd56c baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] Volume volume-270a0ace-7f68-42f8-b371-32e7cc7a8ffc (270a0ace-7f68-42f8-b371-32e7cc7a8ffc): created successfully >2018-08-22 03:43:34.189 39124 INFO cinder.volume.manager [req-e65639c7-d6f8-4e3f-83c9-d781178fd56c baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] Created volume successfully. >2018-08-22 03:43:36.505 39124 INFO cinder.volume.manager [req-5747c6ac-79e1-4d4c-8c58-deb8d5203269 ba91e63d165e4604ba454a8366240af4 2d9e677ec0c0462aad000cacc0a5b736 - default default] Transfer volume completed successfully. >2018-08-22 03:43:42.033 39124 INFO cinder.volume.manager [req-bdb6990e-b1d7-4ea8-948a-2832cc099a38 811140acfe1240d089b03f11d2fbdef4 1e0defaeb8ee45a4a66a329b21a29fee - default default] Deleted volume successfully. >2018-08-22 03:43:43.584 39124 INFO cinder.volume.flows.manager.create_volume [req-fbd82ce2-f7d4-4338-8c45-219bb8438b58 baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] Volume 1b3cef9c-6a2f-4f26-89cc-163049adf41c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1b3cef9c-6a2f-4f26-89cc-163049adf41c'} >2018-08-22 03:43:44.116 39124 INFO cinder.volume.flows.manager.create_volume [req-fbd82ce2-f7d4-4338-8c45-219bb8438b58 baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] Volume volume-1b3cef9c-6a2f-4f26-89cc-163049adf41c (1b3cef9c-6a2f-4f26-89cc-163049adf41c): created successfully >2018-08-22 03:43:44.120 39124 INFO cinder.volume.manager [req-fbd82ce2-f7d4-4338-8c45-219bb8438b58 baec5a3d67dc4cbcb92c8f38a3cc6261 f6e2e0454b424bdd8367028039b39cb8 - default default] Created volume successfully. >2018-08-22 03:43:46.932 39124 INFO cinder.volume.manager [req-fd5160da-feea-454f-8b20-641acc77aaf3 811140acfe1240d089b03f11d2fbdef4 1e0defaeb8ee45a4a66a329b21a29fee - default default] Deleted volume successfully. >2018-08-22 03:43:53.625 39124 INFO cinder.volume.flows.manager.create_volume [req-45f7e17c-a415-417c-b718-1a2a53034aa1 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Volume 1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6'} >2018-08-22 03:43:54.167 39124 INFO cinder.volume.flows.manager.create_volume [req-45f7e17c-a415-417c-b718-1a2a53034aa1 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Volume volume-1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6 (1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6): created successfully >2018-08-22 03:43:54.171 39124 INFO cinder.volume.manager [req-45f7e17c-a415-417c-b718-1a2a53034aa1 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Created volume successfully. >2018-08-22 03:44:00.770 39124 INFO cinder.volume.manager [req-daa57c5c-c819-482a-8361-13eb6a8d4e48 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Attaching volume 1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6 to instance 7b19477c-c579-4820-9144-71265bc74462 at mountpoint /dev/vdb on host None. >2018-08-22 03:44:00.840 39124 INFO cinder.volume.manager [req-daa57c5c-c819-482a-8361-13eb6a8d4e48 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Attach volume completed successfully. >2018-08-22 03:44:01.259 39124 INFO cinder.volume.manager [req-770258b4-de37-43f0-9b10-9968fc64e590 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Detaching volume 1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6 from instance 7b19477c-c579-4820-9144-71265bc74462. >2018-08-22 03:44:01.341 39124 INFO cinder.volume.manager [req-770258b4-de37-43f0-9b10-9968fc64e590 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Detach volume completed successfully. >2018-08-22 03:44:08.644 39124 INFO cinder.volume.manager [req-93c794b1-cb63-4171-a399-4b3144c1ef5e eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Attaching volume 1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6 to instance 7d34f8b4-3d78-49c6-822c-214c08132f2a at mountpoint /dev/vdb on host None. >2018-08-22 03:44:08.718 39124 INFO cinder.volume.manager [req-93c794b1-cb63-4171-a399-4b3144c1ef5e eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Attach volume completed successfully. >2018-08-22 03:44:09.432 39124 INFO cinder.volume.manager [req-46c1acb8-ec35-429f-9c33-e0e79115b8f9 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Detaching volume 1bef09a8-b6a1-4ad9-863d-4b2f745f8bd6 from instance 7d34f8b4-3d78-49c6-822c-214c08132f2a. >2018-08-22 03:44:09.525 39124 INFO cinder.volume.manager [req-46c1acb8-ec35-429f-9c33-e0e79115b8f9 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Detach volume completed successfully. >2018-08-22 03:44:24.604 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 03:44:41.925 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-5939990e-fea5-4c96-94dc-645767311964 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:44:42.266 39124 INFO cinder.volume.manager [req-5939990e-fea5-4c96-94dc-645767311964 eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Copy volume to image completed successfully. >2018-08-22 03:44:45.262 39124 INFO cinder.volume.manager [req-db6c212b-70d4-4490-9611-91253e089f8e eb874fd96ebe42ebb62a6dfb767d1137 85e2846efd56452baff2df7e0d3ae3fb - default default] Deleted volume successfully. >2018-08-22 03:44:52.287 39124 INFO cinder.volume.flows.manager.create_volume [req-ec0758a9-cb63-488e-88ab-3287a8b149f0 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Volume 77206630-70c5-4b09-9f7c-949f533be824: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-77206630-70c5-4b09-9f7c-949f533be824'} >2018-08-22 03:44:52.962 39124 INFO cinder.volume.flows.manager.create_volume [req-ec0758a9-cb63-488e-88ab-3287a8b149f0 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Volume volume-77206630-70c5-4b09-9f7c-949f533be824 (77206630-70c5-4b09-9f7c-949f533be824): created successfully >2018-08-22 03:44:52.966 39124 INFO cinder.volume.manager [req-ec0758a9-cb63-488e-88ab-3287a8b149f0 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Created volume successfully. >2018-08-22 03:45:10.915 39124 INFO cinder.volume.manager [req-767c903d-0384-40bf-9864-488c5f4f578b f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Initialize volume connection completed successfully. >2018-08-22 03:45:15.685 39124 INFO cinder.volume.manager [req-9042c20f-8488-416d-9b78-deab5cbd224e f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Attaching volume 77206630-70c5-4b09-9f7c-949f533be824 to instance 0c0944ce-feee-44f1-bc02-a52a2bef648f at mountpoint /dev/vdb on host None. >2018-08-22 03:45:15.760 39124 INFO cinder.volume.manager [req-9042c20f-8488-416d-9b78-deab5cbd224e f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Attach volume completed successfully. >2018-08-22 03:45:16.919 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0327ed69-5889-451c-9478-f28999bb84f5 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Unable to get remote copy information for volume volume-77206630-70c5-4b09-9f7c-949f533be824 >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova [req-0327ed69-5889-451c-9478-f28999bb84f5 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Failed to notify nova on events: [{'tag': '77206630-70c5-4b09-9f7c-949f533be824', 'name': 'volume-extended', 'server_uuid': '0c0944ce-feee-44f1-bc02-a52a2bef648f'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-0d46b5e3-c88f-4900-acd4-2437195cea5d) >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova return_raw=True) >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-0d46b5e3-c88f-4900-acd4-2437195cea5d) >2018-08-22 03:45:17.646 39124 ERROR cinder.compute.nova >2018-08-22 03:45:17.670 39124 INFO cinder.volume.manager [req-0327ed69-5889-451c-9478-f28999bb84f5 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Extend volume completed successfully. >2018-08-22 03:50:28.412 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b32580ed-11ce-45ec-a781-27355b89c564 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:50:28.865 39124 INFO cinder.volume.manager [req-b32580ed-11ce-45ec-a781-27355b89c564 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Terminate volume connection completed successfully. >2018-08-22 03:50:29.252 39124 INFO cinder.volume.manager [req-95276533-8e60-49ed-a948-47cf46f5e0d6 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Detaching volume 77206630-70c5-4b09-9f7c-949f533be824 from instance 0c0944ce-feee-44f1-bc02-a52a2bef648f. >2018-08-22 03:50:29.340 39124 INFO cinder.volume.manager [req-95276533-8e60-49ed-a948-47cf46f5e0d6 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Detach volume completed successfully. >2018-08-22 03:50:33.487 39124 INFO cinder.volume.manager [req-d806c66e-176e-4da1-a885-fb6e093b2d79 f4322cb622bf4db29731da6ff6b3375a 97cc5701da2b4bd3ad67b869feaf1e37 - default default] Deleted volume successfully. >2018-08-22 03:50:38.532 39124 INFO cinder.volume.flows.manager.create_volume [req-1fc79586-5b9d-44f7-9590-8e4836c04922 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] Volume fad61ba6-33e2-4449-8c06-69f10cef96c6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fad61ba6-33e2-4449-8c06-69f10cef96c6'} >2018-08-22 03:50:39.112 39124 INFO cinder.volume.flows.manager.create_volume [req-1fc79586-5b9d-44f7-9590-8e4836c04922 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] Volume volume-fad61ba6-33e2-4449-8c06-69f10cef96c6 (fad61ba6-33e2-4449-8c06-69f10cef96c6): created successfully >2018-08-22 03:50:39.116 39124 INFO cinder.volume.manager [req-1fc79586-5b9d-44f7-9590-8e4836c04922 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] Created volume successfully. >2018-08-22 03:50:40.754 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-14ade698-f130-4f38-aaa5-079047819fc4 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] Unable to get remote copy information for volume volume-fad61ba6-33e2-4449-8c06-69f10cef96c6 >2018-08-22 03:50:41.060 39124 INFO cinder.volume.manager [req-14ade698-f130-4f38-aaa5-079047819fc4 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] Extend volume completed successfully. >2018-08-22 03:50:43.382 39124 INFO cinder.volume.manager [req-d1a60c2e-bde6-4049-b1e8-5d75b387bcc7 440012b542a341f5b3b8702ab6645ed1 836a3fb9e61c467ebed4407701e164bd - default default] Deleted volume successfully. >2018-08-22 03:50:47.705 39124 INFO cinder.volume.flows.manager.create_volume [req-e8e1efd5-157b-4246-802d-4571d6bba926 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume 10e3d309-bb18-4867-bc2f-4060968b32ee: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-10e3d309-bb18-4867-bc2f-4060968b32ee'} >2018-08-22 03:50:48.260 39124 INFO cinder.volume.flows.manager.create_volume [req-e8e1efd5-157b-4246-802d-4571d6bba926 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume volume-10e3d309-bb18-4867-bc2f-4060968b32ee (10e3d309-bb18-4867-bc2f-4060968b32ee): created successfully >2018-08-22 03:50:48.264 39124 INFO cinder.volume.manager [req-e8e1efd5-157b-4246-802d-4571d6bba926 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Created volume successfully. >2018-08-22 03:50:50.549 39124 INFO cinder.volume.flows.manager.create_volume [req-ef8a364d-9cf6-43b6-bf31-1db4cc9c0bfc 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume 0c9d71b9-fe17-4053-9879-6d4a831d96d6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0c9d71b9-fe17-4053-9879-6d4a831d96d6'} >2018-08-22 03:50:51.077 39124 INFO cinder.volume.flows.manager.create_volume [req-ef8a364d-9cf6-43b6-bf31-1db4cc9c0bfc 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume volume-0c9d71b9-fe17-4053-9879-6d4a831d96d6 (0c9d71b9-fe17-4053-9879-6d4a831d96d6): created successfully >2018-08-22 03:50:51.080 39124 INFO cinder.volume.manager [req-ef8a364d-9cf6-43b6-bf31-1db4cc9c0bfc 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Created volume successfully. >2018-08-22 03:50:53.017 39124 INFO cinder.volume.manager [req-a9d04f02-dc48-4683-ab7d-0309929ce748 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Deleted volume successfully. >2018-08-22 03:50:54.593 39124 INFO cinder.volume.manager [req-fef2c362-a146-438a-b32c-b9df5f4eacc4 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Deleted volume successfully. >2018-08-22 03:50:54.913 39124 INFO cinder.volume.flows.manager.create_volume [req-ab142b78-5c72-4686-a161-2aaa4d32b1c1 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume cfa9c03e-393b-4cc4-8443-ea2d47497020: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cfa9c03e-393b-4cc4-8443-ea2d47497020'} >2018-08-22 03:50:55.527 39124 INFO cinder.volume.flows.manager.create_volume [req-ab142b78-5c72-4686-a161-2aaa4d32b1c1 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume volume-cfa9c03e-393b-4cc4-8443-ea2d47497020 (cfa9c03e-393b-4cc4-8443-ea2d47497020): created successfully >2018-08-22 03:50:55.530 39124 INFO cinder.volume.manager [req-ab142b78-5c72-4686-a161-2aaa4d32b1c1 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Created volume successfully. >2018-08-22 03:50:56.495 39124 INFO cinder.volume.flows.manager.create_volume [req-90d3ffb1-8d8b-414a-afe6-5ed2da116784 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume 29bdbe5f-8b0b-4f86-9813-83b2f7593414: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-29bdbe5f-8b0b-4f86-9813-83b2f7593414', 'source_volid': 'cfa9c03e-393b-4cc4-8443-ea2d47497020', 'source_volstatus': u'available'} >2018-08-22 03:50:58.201 39124 INFO cinder.volume.flows.manager.create_volume [req-90d3ffb1-8d8b-414a-afe6-5ed2da116784 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume volume-29bdbe5f-8b0b-4f86-9813-83b2f7593414 (29bdbe5f-8b0b-4f86-9813-83b2f7593414): created successfully >2018-08-22 03:50:58.204 39124 INFO cinder.volume.manager [req-90d3ffb1-8d8b-414a-afe6-5ed2da116784 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Created volume successfully. >2018-08-22 03:50:59.327 39124 INFO cinder.volume.flows.manager.create_volume [req-e5495693-c65e-4e14-8693-112844e24265 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume 91c02d84-7036-4772-a34b-bb65887f1948: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-91c02d84-7036-4772-a34b-bb65887f1948'} >2018-08-22 03:50:59.982 39124 INFO cinder.volume.flows.manager.create_volume [req-e5495693-c65e-4e14-8693-112844e24265 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume volume-91c02d84-7036-4772-a34b-bb65887f1948 (91c02d84-7036-4772-a34b-bb65887f1948): created successfully >2018-08-22 03:50:59.985 39124 INFO cinder.volume.manager [req-e5495693-c65e-4e14-8693-112844e24265 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Created volume successfully. >2018-08-22 03:51:01.642 39124 INFO cinder.volume.manager [req-1b93955a-9388-452b-bbbf-87b76dca71ea 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Deleted volume successfully. >2018-08-22 03:51:06.070 39124 INFO cinder.volume.manager [req-e173baf6-a174-49fa-ab74-b6ae8f5b6ad7 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Deleted volume successfully. >2018-08-22 03:51:07.486 39124 INFO cinder.volume.flows.manager.create_volume [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume 312be9a4-3055-41c7-bd99-b5d9ba2d310a: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-312be9a4-3055-41c7-bd99-b5d9ba2d310a', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b1d0f090>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 03:51:08.330 39124 INFO cinder.image.image_utils [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 03:51:17.477 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] initialize_connection: Did not find a preferred node for volume volume-312be9a4-3055-41c7-bd99-b5d9ba2d310a. >2018-08-22 03:51:18.106 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 03:51:23.188 39124 INFO cinder.image.image_utils [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 03:51:25.076 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:51:25.578 39124 INFO cinder.volume.flows.manager.create_volume [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume volume-312be9a4-3055-41c7-bd99-b5d9ba2d310a (312be9a4-3055-41c7-bd99-b5d9ba2d310a): created successfully >2018-08-22 03:51:25.582 39124 INFO cinder.volume.manager [req-afa48cea-b3c1-4e33-92e5-276bfc7a6a7b 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Created volume successfully. >2018-08-22 03:51:26.410 39124 INFO cinder.volume.flows.manager.create_volume [req-91d3cd99-5d81-43b7-97f6-c349845593b8 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume a659180d-b42b-4692-842e-e6b1cd3ab2f0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a659180d-b42b-4692-842e-e6b1cd3ab2f0'} >2018-08-22 03:51:26.938 39124 INFO cinder.volume.flows.manager.create_volume [req-91d3cd99-5d81-43b7-97f6-c349845593b8 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Volume volume-a659180d-b42b-4692-842e-e6b1cd3ab2f0 (a659180d-b42b-4692-842e-e6b1cd3ab2f0): created successfully >2018-08-22 03:51:26.941 39124 INFO cinder.volume.manager [req-91d3cd99-5d81-43b7-97f6-c349845593b8 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Created volume successfully. >2018-08-22 03:51:29.104 39124 INFO cinder.volume.manager [req-f86c49ce-c0a6-4f8c-bf11-62125db80396 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Deleted volume successfully. >2018-08-22 03:51:31.145 39124 INFO cinder.volume.manager [req-77f81360-31df-4e99-91bf-a075249a6425 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Deleted volume successfully. >2018-08-22 03:51:32.406 39124 INFO cinder.volume.manager [req-38ed283f-5754-489e-ae8b-e00b23f410e8 8aa9ceb672044c9ea43c4171fc8b5dff ab7a3e7104c44b6a8637eb57fc26abf9 - default default] Deleted volume successfully. >2018-08-22 03:51:41.286 39124 INFO cinder.volume.flows.manager.create_volume [req-7dc02e44-2c9f-41d7-8323-7461bb580ef1 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Volume f815b187-b31f-4da2-9f92-ae197f42bd92: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f815b187-b31f-4da2-9f92-ae197f42bd92'} >2018-08-22 03:51:41.830 39124 INFO cinder.volume.flows.manager.create_volume [req-7dc02e44-2c9f-41d7-8323-7461bb580ef1 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Volume volume-f815b187-b31f-4da2-9f92-ae197f42bd92 (f815b187-b31f-4da2-9f92-ae197f42bd92): created successfully >2018-08-22 03:51:41.836 39124 INFO cinder.volume.manager [req-7dc02e44-2c9f-41d7-8323-7461bb580ef1 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Created volume successfully. >2018-08-22 03:51:43.576 39124 INFO cinder.volume.flows.manager.create_volume [req-be94d185-6092-4637-a078-8643649bd28d 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Volume b330d239-096a-4e35-9bd5-eb602ebafc6f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b330d239-096a-4e35-9bd5-eb602ebafc6f'} >2018-08-22 03:51:44.223 39124 INFO cinder.volume.flows.manager.create_volume [req-be94d185-6092-4637-a078-8643649bd28d 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Volume volume-b330d239-096a-4e35-9bd5-eb602ebafc6f (b330d239-096a-4e35-9bd5-eb602ebafc6f): created successfully >2018-08-22 03:51:44.227 39124 INFO cinder.volume.manager [req-be94d185-6092-4637-a078-8643649bd28d 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Created volume successfully. >2018-08-22 03:51:44.954 39124 INFO cinder.volume.flows.manager.create_volume [req-2137916a-6f09-4a58-9adf-e09282794c41 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Volume e6acb8c0-bec0-4452-8e14-5fe1bfe0d8f4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e6acb8c0-bec0-4452-8e14-5fe1bfe0d8f4'} >2018-08-22 03:51:45.486 39124 INFO cinder.volume.flows.manager.create_volume [req-2137916a-6f09-4a58-9adf-e09282794c41 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Volume volume-e6acb8c0-bec0-4452-8e14-5fe1bfe0d8f4 (e6acb8c0-bec0-4452-8e14-5fe1bfe0d8f4): created successfully >2018-08-22 03:51:45.490 39124 INFO cinder.volume.manager [req-2137916a-6f09-4a58-9adf-e09282794c41 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Created volume successfully. >2018-08-22 03:51:51.879 39124 INFO cinder.volume.manager [req-cf647a29-40a2-4c52-99aa-ff62260452f1 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Deleted volume successfully. >2018-08-22 03:51:51.908 39124 INFO cinder.volume.manager [req-cb5c05f1-c4e0-44aa-95c3-035decee368d 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Deleted volume successfully. >2018-08-22 03:51:52.125 39124 INFO cinder.volume.manager [req-06d920cb-6eb7-4f7f-8377-81d9b3dd67b0 1ddaabc858a446efa49bff446682eb9d e4b37a009df2445499c23da5484b4246 - default default] Deleted volume successfully. >2018-08-22 03:51:56.345 39124 INFO cinder.volume.flows.manager.create_volume [req-66296a83-be26-478e-b952-acf18e837af3 701dded3503648c888fbd814f3e7a077 bb6bc16a3cf64620b0a01f3288a48099 - default default] Volume d46170a1-f5c0-46ba-a0cb-4249f14576ad: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d46170a1-f5c0-46ba-a0cb-4249f14576ad'} >2018-08-22 03:51:58.850 39124 INFO cinder.volume.flows.manager.create_volume [req-66296a83-be26-478e-b952-acf18e837af3 701dded3503648c888fbd814f3e7a077 bb6bc16a3cf64620b0a01f3288a48099 - default default] Volume volume-d46170a1-f5c0-46ba-a0cb-4249f14576ad (d46170a1-f5c0-46ba-a0cb-4249f14576ad): created successfully >2018-08-22 03:51:58.854 39124 INFO cinder.volume.manager [req-66296a83-be26-478e-b952-acf18e837af3 701dded3503648c888fbd814f3e7a077 bb6bc16a3cf64620b0a01f3288a48099 - default default] Created volume successfully. >2018-08-22 03:52:17.146 39124 INFO cinder.volume.manager [req-616128b4-d6b3-47c3-9ba2-9b6c2480ebc2 701dded3503648c888fbd814f3e7a077 bb6bc16a3cf64620b0a01f3288a48099 - default default] Deleted volume successfully. >2018-08-22 03:52:21.677 39124 INFO cinder.volume.flows.manager.create_volume [req-5f2f74f8-8c8a-4aa3-a122-a48947dea452 56d526d9c5ba40b28fb9a9dc6cd1ff45 ca6fed03c12d4dbcac252dfc08c6855b - default default] Volume 7f43cb18-013b-471d-b69d-667b80b73c6b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7f43cb18-013b-471d-b69d-667b80b73c6b'} >2018-08-22 03:52:22.205 39124 INFO cinder.volume.flows.manager.create_volume [req-5f2f74f8-8c8a-4aa3-a122-a48947dea452 56d526d9c5ba40b28fb9a9dc6cd1ff45 ca6fed03c12d4dbcac252dfc08c6855b - default default] Volume volume-7f43cb18-013b-471d-b69d-667b80b73c6b (7f43cb18-013b-471d-b69d-667b80b73c6b): created successfully >2018-08-22 03:52:22.209 39124 INFO cinder.volume.manager [req-5f2f74f8-8c8a-4aa3-a122-a48947dea452 56d526d9c5ba40b28fb9a9dc6cd1ff45 ca6fed03c12d4dbcac252dfc08c6855b - default default] Created volume successfully. >2018-08-22 03:52:24.461 39124 INFO cinder.volume.manager [req-addf2d27-8d71-442d-9112-ae67c41ee3b8 56d526d9c5ba40b28fb9a9dc6cd1ff45 ca6fed03c12d4dbcac252dfc08c6855b - default default] Deleted volume successfully. >2018-08-22 03:53:12.457 39124 INFO cinder.volume.flows.manager.create_volume [req-8430213a-e871-43ea-976c-fc978f2932b2 c9a4cabc8d9f48f28f656f32ad84e6c2 278a25066d75483e9c05d25825d99cce - default default] Volume eb49cdd7-6e78-4bca-8c42-21da3309888b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-eb49cdd7-6e78-4bca-8c42-21da3309888b'} >2018-08-22 03:53:12.984 39124 INFO cinder.volume.flows.manager.create_volume [req-8430213a-e871-43ea-976c-fc978f2932b2 c9a4cabc8d9f48f28f656f32ad84e6c2 278a25066d75483e9c05d25825d99cce - default default] Volume volume-eb49cdd7-6e78-4bca-8c42-21da3309888b (eb49cdd7-6e78-4bca-8c42-21da3309888b): created successfully >2018-08-22 03:53:12.988 39124 INFO cinder.volume.manager [req-8430213a-e871-43ea-976c-fc978f2932b2 c9a4cabc8d9f48f28f656f32ad84e6c2 278a25066d75483e9c05d25825d99cce - default default] Created volume successfully. >2018-08-22 03:53:17.168 39124 INFO cinder.volume.manager [req-255b1455-cbe6-4991-84de-d0e07319e79c c9a4cabc8d9f48f28f656f32ad84e6c2 278a25066d75483e9c05d25825d99cce - default default] Deleted volume successfully. >2018-08-22 03:53:23.762 39124 INFO cinder.volume.flows.manager.create_volume [req-f8ad4018-5d3e-4776-94c0-81c1389e4388 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Volume 4c810465-a625-4820-9839-7485a29c4250: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-4c810465-a625-4820-9839-7485a29c4250', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b21b3cd0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 03:53:24.640 39124 INFO cinder.image.image_utils [req-f8ad4018-5d3e-4776-94c0-81c1389e4388 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 03:53:39.703 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 03:53:44.626 39124 INFO cinder.image.image_utils [req-f8ad4018-5d3e-4776-94c0-81c1389e4388 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 03:53:47.508 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f8ad4018-5d3e-4776-94c0-81c1389e4388 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:53:48.096 39124 INFO cinder.volume.flows.manager.create_volume [req-f8ad4018-5d3e-4776-94c0-81c1389e4388 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Volume volume-4c810465-a625-4820-9839-7485a29c4250 (4c810465-a625-4820-9839-7485a29c4250): created successfully >2018-08-22 03:53:48.100 39124 INFO cinder.volume.manager [req-f8ad4018-5d3e-4776-94c0-81c1389e4388 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Created volume successfully. >2018-08-22 03:53:49.468 39124 INFO cinder.volume.flows.manager.create_volume [req-a5fdf1c3-ba3b-486d-aed0-864812484129 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Volume c13800af-f8ff-4a6e-a9d7-bdeedb3afe48: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c13800af-f8ff-4a6e-a9d7-bdeedb3afe48', 'source_volid': '4c810465-a625-4820-9839-7485a29c4250', 'source_volstatus': u'available'} >2018-08-22 03:53:52.210 39124 INFO cinder.volume.flows.manager.create_volume [req-a5fdf1c3-ba3b-486d-aed0-864812484129 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Volume volume-c13800af-f8ff-4a6e-a9d7-bdeedb3afe48 (c13800af-f8ff-4a6e-a9d7-bdeedb3afe48): created successfully >2018-08-22 03:53:52.214 39124 INFO cinder.volume.manager [req-a5fdf1c3-ba3b-486d-aed0-864812484129 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Created volume successfully. >2018-08-22 03:53:57.680 39124 INFO cinder.volume.manager [req-2eb7a395-419d-4e81-b967-e38ac8beb435 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Deleted volume successfully. >2018-08-22 03:53:58.633 39124 INFO cinder.volume.manager [req-6f8cd676-ab29-4c99-acac-a3e97186fe39 663d21f994624610a9733c642786803b 39aa4d7bc6fd42529faadcd5aa252931 - default default] Deleted volume successfully. >2018-08-22 03:54:03.368 39124 INFO cinder.volume.flows.manager.create_volume [req-1624cd19-0825-44f2-9198-dc5c7beef6c1 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Volume 28683551-c25c-4dd5-8d76-ddabe56eb3a7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-28683551-c25c-4dd5-8d76-ddabe56eb3a7'} >2018-08-22 03:54:04.511 39124 INFO cinder.volume.flows.manager.create_volume [req-1624cd19-0825-44f2-9198-dc5c7beef6c1 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Volume volume-28683551-c25c-4dd5-8d76-ddabe56eb3a7 (28683551-c25c-4dd5-8d76-ddabe56eb3a7): created successfully >2018-08-22 03:54:04.514 39124 INFO cinder.volume.manager [req-1624cd19-0825-44f2-9198-dc5c7beef6c1 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Created volume successfully. >2018-08-22 03:54:05.442 39124 INFO cinder.volume.flows.manager.create_volume [req-d1a71e10-6604-4319-8951-193a8d7ee53b 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Volume 67486be7-c3b5-4633-957d-4330c8367d88: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-67486be7-c3b5-4633-957d-4330c8367d88'} >2018-08-22 03:54:06.363 39124 INFO cinder.volume.flows.manager.create_volume [req-d1a71e10-6604-4319-8951-193a8d7ee53b 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Volume volume-67486be7-c3b5-4633-957d-4330c8367d88 (67486be7-c3b5-4633-957d-4330c8367d88): created successfully >2018-08-22 03:54:06.367 39124 INFO cinder.volume.manager [req-d1a71e10-6604-4319-8951-193a8d7ee53b 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Created volume successfully. >2018-08-22 03:54:07.476 39116 INFO cinder.volume.flows.manager.create_volume [req-e0eb046f-25ed-4dfb-aad5-0e3d14661e5c 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Volume 97b8be32-10d9-4a61-ab10-963b0c5b56b3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-97b8be32-10d9-4a61-ab10-963b0c5b56b3'} >2018-08-22 03:54:08.484 39116 INFO cinder.volume.flows.manager.create_volume [req-e0eb046f-25ed-4dfb-aad5-0e3d14661e5c 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Volume volume-97b8be32-10d9-4a61-ab10-963b0c5b56b3 (97b8be32-10d9-4a61-ab10-963b0c5b56b3): created successfully >2018-08-22 03:54:08.488 39116 INFO cinder.volume.manager [req-e0eb046f-25ed-4dfb-aad5-0e3d14661e5c 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Created volume successfully. >2018-08-22 03:54:12.750 39124 INFO cinder.volume.manager [req-852e7cee-02df-481d-a5ab-abf18ee8679e 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Deleted volume successfully. >2018-08-22 03:54:12.992 39124 INFO cinder.volume.manager [req-b1c2b049-6714-40ea-ae8b-24d6800209b6 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Deleted volume successfully. >2018-08-22 03:54:13.329 39116 INFO cinder.volume.manager [req-9a9717c0-6317-4640-be4b-8068d95a31ec 3e31a91f15a24002a5447b4891b29a42 70bbe352fb1245eaa230506debb4a942 - default default] Deleted volume successfully. >2018-08-22 03:54:33.272 39116 INFO cinder.volume.flows.manager.create_volume [req-55930379-1082-45b5-b6f8-1d925538045c 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Volume 38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e'} >2018-08-22 03:54:34.074 39116 INFO cinder.volume.flows.manager.create_volume [req-55930379-1082-45b5-b6f8-1d925538045c 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Volume volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e (38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e): created successfully >2018-08-22 03:54:34.078 39116 INFO cinder.volume.manager [req-55930379-1082-45b5-b6f8-1d925538045c 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Created volume successfully. >2018-08-22 03:54:35.903 39124 INFO cinder.volume.flows.manager.create_volume [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Volume cab2bdab-8595-45da-b0f5-977a1155005f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cab2bdab-8595-45da-b0f5-977a1155005f'} >2018-08-22 03:54:36.732 39124 INFO cinder.volume.flows.manager.create_volume [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Volume volume-cab2bdab-8595-45da-b0f5-977a1155005f (cab2bdab-8595-45da-b0f5-977a1155005f): created successfully >2018-08-22 03:54:36.736 39124 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Created volume successfully. >2018-08-22 03:54:44.423 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] initialize_connection: Did not find a preferred node for volume volume-cab2bdab-8595-45da-b0f5-977a1155005f. >2018-08-22 03:54:44.645 39124 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Initialize volume connection completed successfully. >2018-08-22 03:54:45.044 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 03:54:49.084 39116 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Initialize volume connection completed successfully. >2018-08-22 03:54:49.469 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 03:54:54.726 39116 INFO cinder.volume.utils [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Volume copy 1024.00 MB at 446.95 MB/s >2018-08-22 03:54:58.472 39124 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Terminate volume connection completed successfully. >2018-08-22 03:54:58.514 39124 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Remove volume export completed successfully. >2018-08-22 03:55:00.363 39116 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:55:00.687 39116 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Terminate volume connection completed successfully. >2018-08-22 03:55:00.719 39116 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Remove volume export completed successfully. >2018-08-22 03:55:01.004 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 03:55:05.557 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Error running SSH command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 03:55:05.560 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 03:55:05.561 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Error running SSH command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 03:55:05.562 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e', '"volume-cab2bdab-8595-45da-b0f5-977a1155005f"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e "volume-cab2bdab-8595-45da-b0f5-977a1155005f" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 03:55:05.563 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Unable to rename the logical volume for volume: 38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-38e2434b-7fa3-48e4-8c3c-eabd2e0fbb1e', '"volume-cab2bdab-8595-45da-b0f5-977a1155005f"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-22 03:55:05.725 39116 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Complete-Migrate volume completed successfully. >2018-08-22 03:55:05.726 39116 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Migrate volume completed successfully. >2018-08-22 03:55:06.228 39116 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Retype volume completed successfully. >2018-08-22 03:55:06.880 39116 INFO cinder.volume.manager [req-67736e63-7b8d-4792-bbf1-61fa2bffd1d8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Deleted volume successfully. >2018-08-22 03:55:08.100 39124 WARNING cinder.quota [req-dae323e4-5d10-4094-8f51-28f6f52301a8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1086082376 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1086082376, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:55:08.101 39124 WARNING cinder.quota [req-dae323e4-5d10-4094-8f51-28f6f52301a8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1086082376 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1086082376, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 03:55:08.454 39124 INFO cinder.volume.manager [req-dae323e4-5d10-4094-8f51-28f6f52301a8 531f17e8c96e49619a965e7ee2fbc290 d00d1a6ab87c40d5adf5a64665c33694 - default default] Deleted volume successfully. >2018-08-22 03:55:29.205 39124 INFO cinder.volume.flows.manager.create_volume [req-cf5f77ab-de5b-4c89-9ec2-051f16b138b3 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Volume 1a0fac0b-5409-431a-abc4-4dbde6504037: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1a0fac0b-5409-431a-abc4-4dbde6504037'} >2018-08-22 03:55:29.752 39124 INFO cinder.volume.flows.manager.create_volume [req-cf5f77ab-de5b-4c89-9ec2-051f16b138b3 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Volume volume-1a0fac0b-5409-431a-abc4-4dbde6504037 (1a0fac0b-5409-431a-abc4-4dbde6504037): created successfully >2018-08-22 03:55:29.756 39124 INFO cinder.volume.manager [req-cf5f77ab-de5b-4c89-9ec2-051f16b138b3 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Created volume successfully. >2018-08-22 03:55:32.823 39124 INFO cinder.volume.manager [req-97ca9543-4751-4064-880b-a6cebeb6f705 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Create snapshot completed successfully >2018-08-22 03:55:36.086 39124 INFO cinder.volume.manager [req-183c1268-c04f-4633-9175-fc2ede390eaa d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Create snapshot completed successfully >2018-08-22 03:55:39.496 39124 INFO cinder.volume.manager [req-6dc4ad18-822d-4997-bdc9-0a3c2fd5e0d7 cb7de0823bc748f7b8c6c8bddc22570b dd3f28933e1f42be9904aed6d9392dc1 - default default] Delete snapshot completed successfully. >2018-08-22 03:55:41.992 39124 INFO cinder.volume.manager [req-2e2e2327-a0a4-43e7-8540-1f29866f7070 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Create snapshot completed successfully >2018-08-22 03:55:44.972 39124 INFO cinder.volume.manager [req-9b07f090-608e-4757-a104-904dfed02ad1 cb7de0823bc748f7b8c6c8bddc22570b dd3f28933e1f42be9904aed6d9392dc1 - default default] Delete snapshot completed successfully. >2018-08-22 03:55:47.489 39124 INFO cinder.volume.manager [req-d83cae45-66e3-4130-b529-79e29cade8aa d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Create snapshot completed successfully >2018-08-22 03:55:50.683 39124 INFO cinder.volume.manager [req-b643eef3-6d1d-4c68-bc6d-d7609441679f cb7de0823bc748f7b8c6c8bddc22570b dd3f28933e1f42be9904aed6d9392dc1 - default default] Delete snapshot completed successfully. >2018-08-22 03:55:53.562 39124 INFO cinder.volume.manager [req-5cbdcf47-e1ea-41a2-9311-78284a5871e0 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Create snapshot completed successfully >2018-08-22 03:55:56.085 39124 INFO cinder.volume.manager [req-e53ba544-c058-4a33-a701-d771e4d05f4e cb7de0823bc748f7b8c6c8bddc22570b dd3f28933e1f42be9904aed6d9392dc1 - default default] Delete snapshot completed successfully. >2018-08-22 03:55:59.137 39124 INFO cinder.volume.manager [req-4070be2a-4c6b-4c51-82d0-54dc6e2328a6 d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Delete snapshot completed successfully. >2018-08-22 03:56:00.656 39124 INFO cinder.volume.manager [req-3c00d9dd-f5d0-418a-a6f4-fdbe4a06040b d6aefd190f6f4b6ea8abac084d93a790 6a8d9f9276ea417e8034f665e02c6ad6 - default default] Deleted volume successfully. >2018-08-22 03:56:05.696 39124 INFO cinder.volume.flows.manager.create_volume [req-8b1c6567-6810-42a8-9661-1d29933f8b48 5dac8aff3c6f4df3ba01d96aded36455 62ba3e9889a64be7bc313c1d850e9ff7 - default default] Volume 50425039-62d3-45b9-b3e2-df23b60ee7a5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-50425039-62d3-45b9-b3e2-df23b60ee7a5'} >2018-08-22 03:56:06.279 39124 INFO cinder.volume.flows.manager.create_volume [req-8b1c6567-6810-42a8-9661-1d29933f8b48 5dac8aff3c6f4df3ba01d96aded36455 62ba3e9889a64be7bc313c1d850e9ff7 - default default] Volume volume-50425039-62d3-45b9-b3e2-df23b60ee7a5 (50425039-62d3-45b9-b3e2-df23b60ee7a5): created successfully >2018-08-22 03:56:06.283 39124 INFO cinder.volume.manager [req-8b1c6567-6810-42a8-9661-1d29933f8b48 5dac8aff3c6f4df3ba01d96aded36455 62ba3e9889a64be7bc313c1d850e9ff7 - default default] Created volume successfully. >2018-08-22 03:56:09.394 39124 INFO cinder.volume.manager [req-5e22fbb6-1d99-4509-aab5-0e77bdb8efda 5dac8aff3c6f4df3ba01d96aded36455 62ba3e9889a64be7bc313c1d850e9ff7 - default default] Create snapshot completed successfully >2018-08-22 03:56:13.779 39124 INFO cinder.volume.manager [req-3ab99f9e-06f3-4155-80b7-260b7d06b4a1 5dac8aff3c6f4df3ba01d96aded36455 62ba3e9889a64be7bc313c1d850e9ff7 - default default] Delete snapshot completed successfully. >2018-08-22 03:56:16.214 39124 INFO cinder.volume.manager [req-b21634ae-0bb0-4ec3-86d4-023fcdc0c566 5dac8aff3c6f4df3ba01d96aded36455 62ba3e9889a64be7bc313c1d850e9ff7 - default default] Deleted volume successfully. >2018-08-22 03:56:20.760 39124 INFO cinder.volume.flows.manager.create_volume [req-a7131bf6-ae9d-491d-a7a5-3826e55c6c4c d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Volume 70db3056-649f-438a-8092-c2f20e98ad60: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-70db3056-649f-438a-8092-c2f20e98ad60'} >2018-08-22 03:56:21.289 39124 INFO cinder.volume.flows.manager.create_volume [req-a7131bf6-ae9d-491d-a7a5-3826e55c6c4c d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Volume volume-70db3056-649f-438a-8092-c2f20e98ad60 (70db3056-649f-438a-8092-c2f20e98ad60): created successfully >2018-08-22 03:56:21.293 39124 INFO cinder.volume.manager [req-a7131bf6-ae9d-491d-a7a5-3826e55c6c4c d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Created volume successfully. >2018-08-22 03:56:26.214 39124 INFO cinder.volume.manager [req-3bc89bbd-fd7d-4d76-85c2-be589f19c8c7 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:56:36.704 39124 INFO cinder.volume.manager [req-ca00b7c6-898c-48bd-a470-113321afd29e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Initialize volume connection completed successfully. >2018-08-22 03:56:59.261 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-ca00b7c6-898c-48bd-a470-113321afd29e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:56:59.573 39124 INFO cinder.volume.manager [req-ca00b7c6-898c-48bd-a470-113321afd29e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Terminate volume connection completed successfully. >2018-08-22 03:56:59.614 39124 INFO cinder.volume.manager [req-ca00b7c6-898c-48bd-a470-113321afd29e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Remove volume export completed successfully. >2018-08-22 03:57:00.860 39124 INFO cinder.volume.manager [req-ca00b7c6-898c-48bd-a470-113321afd29e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Deleted volume successfully. >2018-08-22 03:57:17.256 39124 INFO cinder.volume.manager [req-9806c410-39fe-4787-8bec-0600d37ad324 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Initialize volume connection completed successfully. >2018-08-22 03:57:21.746 39124 INFO cinder.volume.manager [req-3bd4ee64-8aa7-4de3-8947-be506f0470a8 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Attaching volume 70db3056-649f-438a-8092-c2f20e98ad60 to instance efe3ddcb-893b-449b-92e3-b6a9bb3ba7cf at mountpoint /dev/vdb on host None. >2018-08-22 03:57:21.828 39124 INFO cinder.volume.manager [req-3bd4ee64-8aa7-4de3-8947-be506f0470a8 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Attach volume completed successfully. >2018-08-22 03:57:25.522 39124 INFO cinder.volume.manager [req-1bbeaf51-7510-451d-a4c3-8708c63bc03e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:57:28.745 39124 INFO cinder.volume.manager [req-16713901-85df-4f54-b453-e2af9a900437 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:57:32.479 39124 INFO cinder.volume.manager [req-1238a054-f195-43ed-afc9-893dc75cf40f d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:57:35.864 39124 INFO cinder.volume.manager [req-f43cda61-a0b2-4feb-b483-bc2a95a2d308 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:57:40.668 39124 INFO cinder.volume.manager [req-3b6393b3-5a3d-46d3-8a29-8d6b3290343e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:57:44.910 39124 INFO cinder.volume.manager [req-a2bff3fe-8392-4aed-a2e3-25f33c02c706 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:57:57.145 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-69f11183-0e63-4a31-9cdc-c1e5494d8bd4 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:57:57.603 39124 INFO cinder.volume.manager [req-69f11183-0e63-4a31-9cdc-c1e5494d8bd4 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Terminate volume connection completed successfully. >2018-08-22 03:57:57.727 39124 INFO cinder.volume.manager [req-0914d76f-2743-47b7-8d34-6fe707bac3a2 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Detaching volume 70db3056-649f-438a-8092-c2f20e98ad60 from instance efe3ddcb-893b-449b-92e3-b6a9bb3ba7cf. >2018-08-22 03:57:57.811 39124 INFO cinder.volume.manager [req-0914d76f-2743-47b7-8d34-6fe707bac3a2 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Detach volume completed successfully. >2018-08-22 03:58:03.805 39124 INFO cinder.volume.manager [req-5c7c221a-0f7a-4b9a-af74-808273bfd53c d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:58:08.641 39124 INFO cinder.volume.manager [req-48477348-e493-40e5-a72d-3561ad2e599b d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:58:12.476 39124 INFO cinder.volume.manager [req-623e0058-733b-4fdf-a31a-1356e259fe3e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:58:28.209 39124 INFO cinder.volume.manager [req-cfea4e13-d4d1-46f3-a743-d6ab69a30c08 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Initialize volume connection completed successfully. >2018-08-22 03:58:32.733 39124 INFO cinder.volume.manager [req-7b6267c8-2900-488b-9094-b2223b789087 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Attaching volume 70db3056-649f-438a-8092-c2f20e98ad60 to instance 25302bb0-f459-4ace-82e1-c3c0f4c6aa5c at mountpoint /dev/vdb on host None. >2018-08-22 03:58:32.809 39124 INFO cinder.volume.manager [req-7b6267c8-2900-488b-9094-b2223b789087 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Attach volume completed successfully. >2018-08-22 03:58:35.775 39124 INFO cinder.volume.manager [req-494461e4-8926-4320-8e19-427959f3d1d1 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:58:37.784 39124 INFO cinder.volume.manager [req-8163809e-7d93-4bcb-ac89-5ac3663ef891 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:58:40.275 39124 INFO cinder.volume.manager [req-54fb4121-61a1-44f9-9324-b69fbb2d5a48 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:58:42.406 39124 INFO cinder.volume.manager [req-dcf2d37d-21ca-4df3-b2f9-4a06e6d1789c d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:58:44.545 39124 INFO cinder.volume.manager [req-a55cc833-4a86-46fd-b4a2-36020d46c4db d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:58:55.314 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d9f4447f-7246-464a-808d-e2db9c231432 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 03:58:55.589 39124 INFO cinder.volume.manager [req-d9f4447f-7246-464a-808d-e2db9c231432 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Terminate volume connection completed successfully. >2018-08-22 03:58:55.731 39124 INFO cinder.volume.manager [req-a862ddf9-99ba-4dca-9bf0-77409481b222 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Detaching volume 70db3056-649f-438a-8092-c2f20e98ad60 from instance 25302bb0-f459-4ace-82e1-c3c0f4c6aa5c. >2018-08-22 03:58:55.814 39124 INFO cinder.volume.manager [req-a862ddf9-99ba-4dca-9bf0-77409481b222 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Detach volume completed successfully. >2018-08-22 03:58:58.595 39124 INFO cinder.volume.flows.manager.create_volume [req-0947e9d1-9130-464b-aa49-f831ad3552ed d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Volume 35f82695-6539-4935-9499-5d8400e22476: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-35f82695-6539-4935-9499-5d8400e22476'} >2018-08-22 03:58:59.122 39124 INFO cinder.volume.flows.manager.create_volume [req-0947e9d1-9130-464b-aa49-f831ad3552ed d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Volume volume-35f82695-6539-4935-9499-5d8400e22476 (35f82695-6539-4935-9499-5d8400e22476): created successfully >2018-08-22 03:58:59.125 39124 INFO cinder.volume.manager [req-0947e9d1-9130-464b-aa49-f831ad3552ed d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Created volume successfully. >2018-08-22 03:59:01.329 39124 INFO cinder.volume.manager [req-df924883-231d-4658-891f-79481e14abf7 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Create snapshot completed successfully >2018-08-22 03:59:02.473 39124 INFO cinder.volume.flows.manager.create_volume [req-da1567f6-5290-4482-acb9-134d5aa9be7b d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Volume e269dcc8-c3c5-43b2-96bb-1528ba681e79: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': '5f67d23f-ec40-4719-8de5-dfeca27b3bc7', 'volume_name': 'volume-e269dcc8-c3c5-43b2-96bb-1528ba681e79'} >2018-08-22 03:59:50.128 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-da1567f6-5290-4482-acb9-134d5aa9be7b d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Unable to get remote copy information for volume volume-e269dcc8-c3c5-43b2-96bb-1528ba681e79 >2018-08-22 03:59:50.468 39124 INFO cinder.volume.flows.manager.create_volume [req-da1567f6-5290-4482-acb9-134d5aa9be7b d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Volume volume-e269dcc8-c3c5-43b2-96bb-1528ba681e79 (e269dcc8-c3c5-43b2-96bb-1528ba681e79): created successfully >2018-08-22 03:59:50.472 39124 INFO cinder.volume.manager [req-da1567f6-5290-4482-acb9-134d5aa9be7b d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Created volume successfully. >2018-08-22 03:59:52.418 39124 INFO cinder.volume.manager [req-19a7e84c-989f-484a-899d-51d2ac805db5 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Deleted volume successfully. >2018-08-22 03:59:56.197 39124 INFO cinder.volume.manager [req-c0750507-d037-4a92-876c-a2f9f39afd29 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:59:56.416 39124 INFO cinder.volume.manager [req-291f1bd2-9469-4112-858c-268357660042 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Delete snapshot completed successfully. >2018-08-22 03:59:59.076 39124 WARNING cinder.db.sqlalchemy.api [req-6fcd550a-1b2f-4cc7-8089-60fdfe2f235e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-22 03:59:59.107 39124 INFO cinder.volume.manager [req-ad640999-f179-418f-b762-e80a3571aa15 d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Deleted volume successfully. >2018-08-22 03:59:59.372 39124 INFO cinder.volume.manager [req-6fcd550a-1b2f-4cc7-8089-60fdfe2f235e d32b3587b0864fe49f8b2ffcaeba6bdf 00ff4023ac4a499c91547b326aeccf4f - default default] Deleted volume successfully. >2018-08-22 04:00:04.179 39124 INFO cinder.volume.flows.manager.create_volume [req-02a2d59a-cead-4ecc-b255-22304f7c9bc2 a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Volume 39b58295-9908-4bb7-968f-9b428ea9a389: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-39b58295-9908-4bb7-968f-9b428ea9a389'} >2018-08-22 04:00:04.708 39124 INFO cinder.volume.flows.manager.create_volume [req-02a2d59a-cead-4ecc-b255-22304f7c9bc2 a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Volume volume-39b58295-9908-4bb7-968f-9b428ea9a389 (39b58295-9908-4bb7-968f-9b428ea9a389): created successfully >2018-08-22 04:00:04.712 39124 INFO cinder.volume.manager [req-02a2d59a-cead-4ecc-b255-22304f7c9bc2 a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Created volume successfully. >2018-08-22 04:00:08.028 39124 INFO cinder.volume.manager [req-96239a63-c7b3-4329-afb9-48de9edbc52e a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Create snapshot completed successfully >2018-08-22 04:00:13.739 39124 INFO cinder.volume.manager [req-45fe75b4-35ee-4351-b792-fbd3b465eddf a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Create snapshot completed successfully >2018-08-22 04:00:16.806 39124 INFO cinder.volume.manager [req-2ab3e6fb-b9d5-4b7e-9e6d-7113583ce53b a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Create snapshot completed successfully >2018-08-22 04:00:23.324 39124 INFO cinder.volume.manager [req-bec8dd44-0305-4bcb-abec-d4180f32b90c a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Delete snapshot completed successfully. >2018-08-22 04:00:23.543 39124 INFO cinder.volume.manager [req-68810c8e-c04d-4467-8f1c-137dfb54bab6 a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Delete snapshot completed successfully. >2018-08-22 04:00:23.777 39124 INFO cinder.volume.manager [req-5589d631-0934-405e-b9e3-7daec65ec732 a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Delete snapshot completed successfully. >2018-08-22 04:00:25.361 39124 INFO cinder.volume.manager [req-f79879a0-5b8d-4fbc-8e84-da2bc54debb9 a255da4b36db4d189aab64ba81f42c6a a846fec95c364824a9e903aa5720cd75 - default default] Deleted volume successfully. >2018-08-22 04:00:30.924 39124 INFO cinder.volume.flows.manager.create_volume [req-bbaa77b5-944e-42fa-86fc-e22e22fe50eb 48529a9db0e14efd88a7fbf248eaa9e1 2feaa821a7a24d78b6120c7c942dfd33 - default default] Volume c7d4aaef-f504-464c-a3fb-924c13c843d3: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-c7d4aaef-f504-464c-a3fb-924c13c843d3'} >2018-08-22 04:00:31.543 39124 INFO cinder.volume.flows.manager.create_volume [req-bbaa77b5-944e-42fa-86fc-e22e22fe50eb 48529a9db0e14efd88a7fbf248eaa9e1 2feaa821a7a24d78b6120c7c942dfd33 - default default] Volume volume-c7d4aaef-f504-464c-a3fb-924c13c843d3 (c7d4aaef-f504-464c-a3fb-924c13c843d3): created successfully >2018-08-22 04:00:31.546 39124 INFO cinder.volume.manager [req-bbaa77b5-944e-42fa-86fc-e22e22fe50eb 48529a9db0e14efd88a7fbf248eaa9e1 2feaa821a7a24d78b6120c7c942dfd33 - default default] Created volume successfully. >2018-08-22 04:00:34.161 39124 INFO cinder.volume.manager [req-1273b2b7-ce69-43e5-857a-5595d69c773c 48529a9db0e14efd88a7fbf248eaa9e1 2feaa821a7a24d78b6120c7c942dfd33 - default default] Create snapshot completed successfully >2018-08-22 04:00:37.751 39124 INFO cinder.volume.manager [req-e2043e9c-8f82-4a71-a87d-8bdc89213e34 48529a9db0e14efd88a7fbf248eaa9e1 2feaa821a7a24d78b6120c7c942dfd33 - default default] Delete snapshot completed successfully. >2018-08-22 04:00:40.058 39124 INFO cinder.volume.manager [req-7ca8166a-e4f0-430c-a430-865403f2f825 48529a9db0e14efd88a7fbf248eaa9e1 2feaa821a7a24d78b6120c7c942dfd33 - default default] Deleted volume successfully. >2018-08-22 04:01:02.675 39124 INFO cinder.volume.flows.manager.create_volume [req-94b85561-def9-4af4-b4fe-7986f2634c0f a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume 6744ada0-343f-40d0-b43a-5023a9752d3e: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-6744ada0-343f-40d0-b43a-5023a9752d3e', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b203be10>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 04:01:03.976 39124 INFO cinder.image.image_utils [req-94b85561-def9-4af4-b4fe-7986f2634c0f a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Image download 12.00 MB at 9.25 MB/s >2018-08-22 04:01:16.487 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 04:01:21.442 39124 INFO cinder.image.image_utils [req-94b85561-def9-4af4-b4fe-7986f2634c0f a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 04:01:24.109 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-94b85561-def9-4af4-b4fe-7986f2634c0f a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 04:01:27.666 39124 INFO cinder.volume.flows.manager.create_volume [req-94b85561-def9-4af4-b4fe-7986f2634c0f a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume volume-6744ada0-343f-40d0-b43a-5023a9752d3e (6744ada0-343f-40d0-b43a-5023a9752d3e): created successfully >2018-08-22 04:01:27.669 39124 INFO cinder.volume.manager [req-94b85561-def9-4af4-b4fe-7986f2634c0f a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Created volume successfully. >2018-08-22 04:01:28.718 39124 INFO cinder.volume.flows.manager.create_volume [req-d88ca37f-ac9e-46ab-a185-8f12975d03af a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume e46370de-9ef7-408b-ba80-17dfb80dd725: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e46370de-9ef7-408b-ba80-17dfb80dd725', 'source_volid': '6744ada0-343f-40d0-b43a-5023a9752d3e', 'source_volstatus': u'available'} >2018-08-22 04:01:31.304 39124 INFO cinder.volume.flows.manager.create_volume [req-d88ca37f-ac9e-46ab-a185-8f12975d03af a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume volume-e46370de-9ef7-408b-ba80-17dfb80dd725 (e46370de-9ef7-408b-ba80-17dfb80dd725): created successfully >2018-08-22 04:01:31.308 39124 INFO cinder.volume.manager [req-d88ca37f-ac9e-46ab-a185-8f12975d03af a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Created volume successfully. >2018-08-22 04:01:32.347 39124 INFO cinder.volume.flows.manager.create_volume [req-21baf813-993c-4a9d-b358-9672bf721f99 a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume 5fdbc4bb-432b-4091-846c-4338424f0399: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5fdbc4bb-432b-4091-846c-4338424f0399'} >2018-08-22 04:01:33.263 39124 INFO cinder.volume.flows.manager.create_volume [req-21baf813-993c-4a9d-b358-9672bf721f99 a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume volume-5fdbc4bb-432b-4091-846c-4338424f0399 (5fdbc4bb-432b-4091-846c-4338424f0399): created successfully >2018-08-22 04:01:33.267 39124 INFO cinder.volume.manager [req-21baf813-993c-4a9d-b358-9672bf721f99 a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Created volume successfully. >2018-08-22 04:01:33.706 39124 INFO cinder.volume.flows.manager.create_volume [req-0bc86fd4-7e11-43cb-bbb9-7fca152aa33e a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume 958f4000-5492-4315-9680-6c6dec71e624: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-958f4000-5492-4315-9680-6c6dec71e624', 'source_volid': '5fdbc4bb-432b-4091-846c-4338424f0399', 'source_volstatus': u'available'} >2018-08-22 04:01:46.018 39124 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 0.29 sec >2018-08-22 04:01:50.371 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0bc86fd4-7e11-43cb-bbb9-7fca152aa33e a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Unable to get remote copy information for volume volume-958f4000-5492-4315-9680-6c6dec71e624 >2018-08-22 04:01:50.814 39124 INFO cinder.volume.flows.manager.create_volume [req-0bc86fd4-7e11-43cb-bbb9-7fca152aa33e a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Volume volume-958f4000-5492-4315-9680-6c6dec71e624 (958f4000-5492-4315-9680-6c6dec71e624): created successfully >2018-08-22 04:01:50.817 39124 INFO cinder.volume.manager [req-0bc86fd4-7e11-43cb-bbb9-7fca152aa33e a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Created volume successfully. >2018-08-22 04:01:55.970 39124 INFO cinder.volume.manager [req-2afc8211-67de-4019-aacc-f76a0cab9a8e a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Deleted volume successfully. >2018-08-22 04:01:56.017 39124 INFO cinder.volume.manager [req-660c4b9a-b0a8-405f-8801-6793ef2e36f4 a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Deleted volume successfully. >2018-08-22 04:01:56.446 39124 INFO cinder.volume.manager [req-37d85cbd-bfc5-4cd5-aa52-564f51d4cf23 a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Deleted volume successfully. >2018-08-22 04:01:56.718 39124 INFO cinder.volume.manager [req-146fb1fe-e4e4-471c-a281-84cb2ca6ebf6 a02b042e954642b3a753f1823156c76c e187c8caa64e46b7bdd12aae07c298ed - default default] Deleted volume successfully. >2018-08-22 04:02:00.393 39124 INFO cinder.volume.flows.manager.create_volume [req-9ec48368-12b0-4a9e-a68a-71c40b59c7c7 3f53fa7be24d4eaa8edab2efac034902 ab077addc5504bf5b259b8a1bfc9543a - default default] Volume 84d7d5f2-a034-4904-8119-6bad3de6d1bf: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-84d7d5f2-a034-4904-8119-6bad3de6d1bf'} >2018-08-22 04:02:01.318 39124 INFO cinder.volume.flows.manager.create_volume [req-9ec48368-12b0-4a9e-a68a-71c40b59c7c7 3f53fa7be24d4eaa8edab2efac034902 ab077addc5504bf5b259b8a1bfc9543a - default default] Volume volume-84d7d5f2-a034-4904-8119-6bad3de6d1bf (84d7d5f2-a034-4904-8119-6bad3de6d1bf): created successfully >2018-08-22 04:02:01.322 39124 INFO cinder.volume.manager [req-9ec48368-12b0-4a9e-a68a-71c40b59c7c7 3f53fa7be24d4eaa8edab2efac034902 ab077addc5504bf5b259b8a1bfc9543a - default default] Created volume successfully. >2018-08-22 04:02:04.550 39124 INFO cinder.volume.manager [req-3cdd750c-3315-4127-b633-2d4c96b4322c 3f53fa7be24d4eaa8edab2efac034902 ab077addc5504bf5b259b8a1bfc9543a - default default] Deleted volume successfully. >2018-08-22 04:02:28.065 39124 INFO cinder.volume.flows.manager.create_volume [req-84fb9ab9-ca01-4d02-a136-78594e1d393f 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] Volume 1c865bc8-ad00-4d32-bc16-1166f320aa20: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1c865bc8-ad00-4d32-bc16-1166f320aa20'} >2018-08-22 04:02:28.686 39124 INFO cinder.volume.flows.manager.create_volume [req-84fb9ab9-ca01-4d02-a136-78594e1d393f 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] Volume volume-1c865bc8-ad00-4d32-bc16-1166f320aa20 (1c865bc8-ad00-4d32-bc16-1166f320aa20): created successfully >2018-08-22 04:02:28.690 39124 INFO cinder.volume.manager [req-84fb9ab9-ca01-4d02-a136-78594e1d393f 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] Created volume successfully. >2018-08-22 04:02:31.554 39124 INFO cinder.volume.manager [req-61dda2e4-2f2c-4636-a5f9-637f0128ad8c 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] Deleted volume successfully. >2018-08-22 04:02:31.898 39124 INFO cinder.volume.flows.manager.create_volume [req-e20948b6-bb8f-412c-9fc4-afe563ed467c 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] Volume 5d6bbfc4-9677-4aa0-b017-2220b90cf8d6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5d6bbfc4-9677-4aa0-b017-2220b90cf8d6'} >2018-08-22 04:02:32.482 39124 INFO cinder.volume.flows.manager.create_volume [req-e20948b6-bb8f-412c-9fc4-afe563ed467c 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] Volume volume-5d6bbfc4-9677-4aa0-b017-2220b90cf8d6 (5d6bbfc4-9677-4aa0-b017-2220b90cf8d6): created successfully >2018-08-22 04:02:32.486 39124 INFO cinder.volume.manager [req-e20948b6-bb8f-412c-9fc4-afe563ed467c 74df0e93f4994c5aa28cc781859cd7ac b11ea2d9663d4b519ec334462bcb124a - default default] Created volume successfully. >2018-08-22 04:02:33.965 39124 INFO cinder.volume.manager [req-ca119f6a-eab1-457e-8d8b-cc2751b7a2d3 59e00019d6974b9b8fc5c3ae73f0bcf8 f50674cb267f4f7d932d163e34ca0dff - default default] Transfer volume completed successfully. >2018-08-22 04:02:36.005 39124 INFO cinder.volume.manager [req-fe87ed55-4d7f-4960-b232-24a8afa176c0 83b8c199b9f84b02805d556f710efb49 907d782666f84108bf1d0d42ca39af34 - default default] Deleted volume successfully. >2018-08-22 04:02:45.603 39124 INFO cinder.volume.flows.manager.create_volume [req-68541dbc-d97d-4009-9d30-612160d3f962 2c665262ab1d4f0cb5a145ad1df07073 b863e7cfc7ce4fe7941b3dba7ecffbd6 - default default] Volume 09915e0c-48d3-4161-893b-2c62b1caa656: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-09915e0c-48d3-4161-893b-2c62b1caa656'} >2018-08-22 04:02:46.147 39124 INFO cinder.volume.flows.manager.create_volume [req-68541dbc-d97d-4009-9d30-612160d3f962 2c665262ab1d4f0cb5a145ad1df07073 b863e7cfc7ce4fe7941b3dba7ecffbd6 - default default] Volume volume-09915e0c-48d3-4161-893b-2c62b1caa656 (09915e0c-48d3-4161-893b-2c62b1caa656): created successfully >2018-08-22 04:02:46.151 39124 INFO cinder.volume.manager [req-68541dbc-d97d-4009-9d30-612160d3f962 2c665262ab1d4f0cb5a145ad1df07073 b863e7cfc7ce4fe7941b3dba7ecffbd6 - default default] Created volume successfully. >2018-08-22 04:02:51.603 39124 INFO cinder.volume.manager [req-f11e90d6-7b12-49c9-b050-82ff70672cd3 2c665262ab1d4f0cb5a145ad1df07073 b863e7cfc7ce4fe7941b3dba7ecffbd6 - default default] Deleted volume successfully. >2018-08-22 04:02:58.861 39124 INFO cinder.volume.flows.manager.create_volume [req-bdc5234d-33e7-44b3-a148-e4726e22fc91 c4f2bad0dc634669bac8c9b7270ba7a0 5034fc8b6aca4e4782bfbc181d38e1cd - default default] Volume 84322055-99b4-4741-aa74-7ed097a0ba85: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-84322055-99b4-4741-aa74-7ed097a0ba85'} >2018-08-22 04:02:59.389 39124 INFO cinder.volume.flows.manager.create_volume [req-bdc5234d-33e7-44b3-a148-e4726e22fc91 c4f2bad0dc634669bac8c9b7270ba7a0 5034fc8b6aca4e4782bfbc181d38e1cd - default default] Volume volume-84322055-99b4-4741-aa74-7ed097a0ba85 (84322055-99b4-4741-aa74-7ed097a0ba85): created successfully >2018-08-22 04:02:59.392 39124 INFO cinder.volume.manager [req-bdc5234d-33e7-44b3-a148-e4726e22fc91 c4f2bad0dc634669bac8c9b7270ba7a0 5034fc8b6aca4e4782bfbc181d38e1cd - default default] Created volume successfully. >2018-08-22 04:03:02.216 39124 INFO cinder.volume.manager [req-8af3c682-1297-453a-8dba-168680b5b7ea c4f2bad0dc634669bac8c9b7270ba7a0 5034fc8b6aca4e4782bfbc181d38e1cd - default default] Create snapshot completed successfully >2018-08-22 04:03:06.043 39124 INFO cinder.volume.manager [req-335aae3f-a3a6-4d83-a0ed-a7e668700fcf c4f2bad0dc634669bac8c9b7270ba7a0 5034fc8b6aca4e4782bfbc181d38e1cd - default default] Delete snapshot completed successfully. >2018-08-22 04:03:10.850 39124 INFO cinder.volume.manager [req-4e30e020-1594-4b4a-989c-b99e18ace6af c4f2bad0dc634669bac8c9b7270ba7a0 5034fc8b6aca4e4782bfbc181d38e1cd - default default] Deleted volume successfully. >2018-08-22 06:29:27.924 39124 INFO cinder.volume.manager [req-68de7a57-c982-47c0-abfd-acfcc3cbd242 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-1559081957: creating >2018-08-22 06:29:27.947 39124 INFO cinder.volume.manager [req-68de7a57-c982-47c0-abfd-acfcc3cbd242 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-1559081957: created successfully >2018-08-22 06:29:27.967 39124 INFO cinder.volume.manager [req-68de7a57-c982-47c0-abfd-acfcc3cbd242 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Create group completed successfully. >2018-08-22 06:29:29.003 39124 INFO cinder.volume.flows.manager.create_volume [req-5e4d8d25-c033-4b6e-9571-013293194199 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume 94696c5a-14d4-4bf5-8bdf-c5621299396e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-94696c5a-14d4-4bf5-8bdf-c5621299396e'} >2018-08-22 06:29:29.672 39124 INFO cinder.volume.flows.manager.create_volume [req-5e4d8d25-c033-4b6e-9571-013293194199 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume volume-94696c5a-14d4-4bf5-8bdf-c5621299396e (94696c5a-14d4-4bf5-8bdf-c5621299396e): created successfully >2018-08-22 06:29:29.676 39124 INFO cinder.volume.manager [req-5e4d8d25-c033-4b6e-9571-013293194199 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Created volume successfully. >2018-08-22 06:29:31.748 39124 INFO cinder.volume.manager [req-80a77294-5142-45db-b656-e770ce61b498 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] GroupSnapshot a59bb46d-3005-4c08-896c-fbd94ba4ccaf: creating. >2018-08-22 06:29:34.220 39124 INFO cinder.volume.manager [req-80a77294-5142-45db-b656-e770ce61b498 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] group_snapshot a59bb46d-3005-4c08-896c-fbd94ba4ccaf: created successfully >2018-08-22 06:29:35.180 39124 INFO cinder.volume.manager [req-45f5b21a-9f59-4e75-8d4b-23f5341a4089 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] group_snapshot a59bb46d-3005-4c08-896c-fbd94ba4ccaf: deleting >2018-08-22 06:29:36.978 39124 WARNING cinder.quota [req-45f5b21a-9f59-4e75-8d4b-23f5341a4089 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-862721771 is set by the default quota flag: quota_snapshots_tempest-volume-type-862721771, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:37.008 39124 INFO cinder.volume.manager [req-45f5b21a-9f59-4e75-8d4b-23f5341a4089 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] group_snapshot a59bb46d-3005-4c08-896c-fbd94ba4ccaf: deleted successfully >2018-08-22 06:29:38.339 39124 WARNING cinder.quota [req-00315dcf-2a30-449b-83ba-6b5845f37438 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-862721771 is set by the default quota flag: quota_gigabytes_tempest-volume-type-862721771, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:38.340 39124 WARNING cinder.quota [req-00315dcf-2a30-449b-83ba-6b5845f37438 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-862721771 is set by the default quota flag: quota_volumes_tempest-volume-type-862721771, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:38.615 39124 INFO cinder.volume.manager [req-00315dcf-2a30-449b-83ba-6b5845f37438 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Delete group completed successfully. >2018-08-22 06:29:39.109 39124 INFO cinder.volume.manager [req-9aa31844-9759-4884-a464-1470def63616 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-1662027231: creating >2018-08-22 06:29:39.131 39124 INFO cinder.volume.manager [req-9aa31844-9759-4884-a464-1470def63616 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-1662027231: created successfully >2018-08-22 06:29:39.154 39124 INFO cinder.volume.manager [req-9aa31844-9759-4884-a464-1470def63616 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Create group completed successfully. >2018-08-22 06:29:39.539 39124 INFO cinder.volume.flows.manager.create_volume [req-97863d1b-ddde-4cdd-ab98-a77c439e09f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume 495b1405-6e8e-4365-b125-dbe9ee6a1d55: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-495b1405-6e8e-4365-b125-dbe9ee6a1d55'} >2018-08-22 06:29:40.103 39124 INFO cinder.volume.flows.manager.create_volume [req-97863d1b-ddde-4cdd-ab98-a77c439e09f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume volume-495b1405-6e8e-4365-b125-dbe9ee6a1d55 (495b1405-6e8e-4365-b125-dbe9ee6a1d55): created successfully >2018-08-22 06:29:40.107 39124 INFO cinder.volume.manager [req-97863d1b-ddde-4cdd-ab98-a77c439e09f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Created volume successfully. >2018-08-22 06:29:41.722 39124 WARNING cinder.quota [req-1c17f8f1-2b5d-4cff-8d0f-4fdd4566e217 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1043202176 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1043202176, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:41.723 39124 WARNING cinder.quota [req-1c17f8f1-2b5d-4cff-8d0f-4fdd4566e217 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1043202176 is set by the default quota flag: quota_volumes_tempest-volume-type-1043202176, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:42.006 39124 INFO cinder.volume.manager [req-1c17f8f1-2b5d-4cff-8d0f-4fdd4566e217 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Delete group completed successfully. >2018-08-22 06:29:42.303 39124 INFO cinder.volume.manager [req-921e2129-17bb-4de2-8c26-01b7fa1f59f1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-719103304: creating >2018-08-22 06:29:42.324 39124 INFO cinder.volume.manager [req-921e2129-17bb-4de2-8c26-01b7fa1f59f1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-719103304: created successfully >2018-08-22 06:29:42.344 39124 INFO cinder.volume.manager [req-921e2129-17bb-4de2-8c26-01b7fa1f59f1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Create group completed successfully. >2018-08-22 06:29:42.740 39124 INFO cinder.volume.flows.manager.create_volume [req-9e503614-2e4f-4fb3-92b5-b550ed31cde1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume f791a3c0-7087-42aa-b4f7-439a2aeb6b3e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f791a3c0-7087-42aa-b4f7-439a2aeb6b3e'} >2018-08-22 06:29:43.290 39124 INFO cinder.volume.flows.manager.create_volume [req-9e503614-2e4f-4fb3-92b5-b550ed31cde1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume volume-f791a3c0-7087-42aa-b4f7-439a2aeb6b3e (f791a3c0-7087-42aa-b4f7-439a2aeb6b3e): created successfully >2018-08-22 06:29:43.293 39124 INFO cinder.volume.manager [req-9e503614-2e4f-4fb3-92b5-b550ed31cde1 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Created volume successfully. >2018-08-22 06:29:43.957 39124 INFO cinder.volume.manager [req-110f6d6f-ff02-4bfd-8b53-81f325287dd4 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] GroupSnapshot bb4bf702-ad52-4cd4-8366-4e43a16bb674: creating. >2018-08-22 06:29:45.970 39124 INFO cinder.volume.manager [req-110f6d6f-ff02-4bfd-8b53-81f325287dd4 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] group_snapshot bb4bf702-ad52-4cd4-8366-4e43a16bb674: created successfully >2018-08-22 06:29:49.019 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-cd73356b-2a27-41bb-82a5-6cd7396ee12d b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Updating status for CG: 2a8935db-b33f-4085-9fb7-a226d482a31e. >2018-08-22 06:29:49.125 39124 INFO cinder.volume.manager [req-cd73356b-2a27-41bb-82a5-6cd7396ee12d b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Create group from source-snapshot-bb4bf702-ad52-4cd4-8366-4e43a16bb674 completed successfully. >2018-08-22 06:29:51.396 39124 WARNING cinder.quota [req-023379e0-7125-4c83-9e65-aa1d8d6ae7c2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-658048885 is set by the default quota flag: quota_volumes_tempest-volume-type-658048885, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:51.397 39124 WARNING cinder.quota [req-023379e0-7125-4c83-9e65-aa1d8d6ae7c2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-658048885 is set by the default quota flag: quota_gigabytes_tempest-volume-type-658048885, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:51.717 39124 INFO cinder.volume.manager [req-023379e0-7125-4c83-9e65-aa1d8d6ae7c2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Delete group completed successfully. >2018-08-22 06:29:52.522 39124 INFO cinder.volume.manager [req-6681357e-3dc7-4fc4-8fb1-94dd972f771d b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] group_snapshot bb4bf702-ad52-4cd4-8366-4e43a16bb674: deleting >2018-08-22 06:29:55.047 39124 WARNING cinder.quota [req-6681357e-3dc7-4fc4-8fb1-94dd972f771d b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-658048885 is set by the default quota flag: quota_snapshots_tempest-volume-type-658048885, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:29:55.078 39124 INFO cinder.volume.manager [req-6681357e-3dc7-4fc4-8fb1-94dd972f771d b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] group_snapshot bb4bf702-ad52-4cd4-8366-4e43a16bb674: deleted successfully >2018-08-22 06:29:57.325 39124 INFO cinder.volume.manager [req-d5b2a93a-bb65-499a-9eb1-3446386f16ce b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Delete group completed successfully. >2018-08-22 06:29:57.624 39124 INFO cinder.volume.manager [req-b812838f-eaa0-44be-906a-bd9121be79f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-1328623804: creating >2018-08-22 06:29:57.646 39124 INFO cinder.volume.manager [req-b812838f-eaa0-44be-906a-bd9121be79f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Group tempest-CG-1328623804: created successfully >2018-08-22 06:29:57.668 39124 INFO cinder.volume.manager [req-b812838f-eaa0-44be-906a-bd9121be79f2 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Create group completed successfully. >2018-08-22 06:29:58.144 39124 INFO cinder.volume.flows.manager.create_volume [req-a0b11437-8373-4d56-b0f9-ecb63f7420aa b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume a263cbfd-69af-4099-8ba4-c87aba654112: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a263cbfd-69af-4099-8ba4-c87aba654112'} >2018-08-22 06:29:58.698 39124 INFO cinder.volume.flows.manager.create_volume [req-a0b11437-8373-4d56-b0f9-ecb63f7420aa b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Volume volume-a263cbfd-69af-4099-8ba4-c87aba654112 (a263cbfd-69af-4099-8ba4-c87aba654112): created successfully >2018-08-22 06:29:58.701 39124 INFO cinder.volume.manager [req-a0b11437-8373-4d56-b0f9-ecb63f7420aa b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Created volume successfully. >2018-08-22 06:30:01.442 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-7985b6e8-a635-4a63-8418-e7c118a5406b b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Updating status for CG: 21ccccd5-ac7d-46f7-bddb-7fd8b0d7c397. >2018-08-22 06:30:01.533 39124 INFO cinder.volume.manager [req-7985b6e8-a635-4a63-8418-e7c118a5406b b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Create group from source-group-9b62d8dd-53ad-4d0a-b461-698325704155 completed successfully. >2018-08-22 06:30:06.015 39124 WARNING cinder.quota [req-8f57f7d5-1398-40b9-a1bc-7eb675eb4a85 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1548273252 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1548273252, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:30:06.017 39124 WARNING cinder.quota [req-8f57f7d5-1398-40b9-a1bc-7eb675eb4a85 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1548273252 is set by the default quota flag: quota_volumes_tempest-volume-type-1548273252, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:30:06.295 39124 INFO cinder.volume.manager [req-8f57f7d5-1398-40b9-a1bc-7eb675eb4a85 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Delete group completed successfully. >2018-08-22 06:30:08.006 39124 INFO cinder.volume.manager [req-a7cbfe8f-2cfd-4d54-a89b-1cb9de631571 b894683dbc8b4a85a76e3a7616306659 e900ac0f1a00486d9de14b79409c4388 - default default] Delete group completed successfully. >2018-08-22 06:31:19.852 39124 INFO cinder.volume.flows.manager.create_volume [req-01c469f7-4bc4-4b43-95fb-ed8fe91d4bc0 b30a7d2131ce48149218bb552595e2d4 ed789a3dee02492fb8047c840a18ebcc - default default] Volume 0fe72893-bd61-484c-a23a-212aa5d16db7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0fe72893-bd61-484c-a23a-212aa5d16db7'} >2018-08-22 06:31:20.383 39124 INFO cinder.volume.flows.manager.create_volume [req-01c469f7-4bc4-4b43-95fb-ed8fe91d4bc0 b30a7d2131ce48149218bb552595e2d4 ed789a3dee02492fb8047c840a18ebcc - default default] Volume volume-0fe72893-bd61-484c-a23a-212aa5d16db7 (0fe72893-bd61-484c-a23a-212aa5d16db7): created successfully >2018-08-22 06:31:20.387 39124 INFO cinder.volume.manager [req-01c469f7-4bc4-4b43-95fb-ed8fe91d4bc0 b30a7d2131ce48149218bb552595e2d4 ed789a3dee02492fb8047c840a18ebcc - default default] Created volume successfully. >2018-08-22 06:31:23.327 39124 INFO cinder.volume.manager [req-f42daf23-a159-440d-b37b-5d0e84a6969a b30a7d2131ce48149218bb552595e2d4 ed789a3dee02492fb8047c840a18ebcc - default default] Deleted volume successfully. >2018-08-22 06:31:32.501 39124 INFO cinder.volume.flows.manager.create_volume [req-7d4f0673-30ca-4e5e-a6ff-95c44b5de4a0 1e01cbe6e0684522904a6eb4f66eef86 94a5a5633fca40b9be7db04938ad26e6 - default default] Volume 19cf028b-5a72-4ac1-b3bc-4419a31795de: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-19cf028b-5a72-4ac1-b3bc-4419a31795de'} >2018-08-22 06:31:33.047 39124 INFO cinder.volume.flows.manager.create_volume [req-7d4f0673-30ca-4e5e-a6ff-95c44b5de4a0 1e01cbe6e0684522904a6eb4f66eef86 94a5a5633fca40b9be7db04938ad26e6 - default default] Volume volume-19cf028b-5a72-4ac1-b3bc-4419a31795de (19cf028b-5a72-4ac1-b3bc-4419a31795de): created successfully >2018-08-22 06:31:33.050 39124 INFO cinder.volume.manager [req-7d4f0673-30ca-4e5e-a6ff-95c44b5de4a0 1e01cbe6e0684522904a6eb4f66eef86 94a5a5633fca40b9be7db04938ad26e6 - default default] Created volume successfully. >2018-08-22 06:31:36.962 39124 WARNING cinder.quota [req-b3e3863d-b092-4073-bb23-8f47dc5355b3 1e01cbe6e0684522904a6eb4f66eef86 94a5a5633fca40b9be7db04938ad26e6 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-36922382 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-36922382, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:31:36.964 39124 WARNING cinder.quota [req-b3e3863d-b092-4073-bb23-8f47dc5355b3 1e01cbe6e0684522904a6eb4f66eef86 94a5a5633fca40b9be7db04938ad26e6 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-36922382 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-36922382, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:31:37.277 39124 INFO cinder.volume.manager [req-b3e3863d-b092-4073-bb23-8f47dc5355b3 1e01cbe6e0684522904a6eb4f66eef86 94a5a5633fca40b9be7db04938ad26e6 - default default] Deleted volume successfully. >2018-08-22 06:31:45.800 39124 INFO cinder.volume.flows.manager.create_volume [req-ae4437f2-14fd-4e66-9a8e-b20f45eac27e b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] Volume fb72c65a-42ae-439d-9bc3-d35be43cb707: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fb72c65a-42ae-439d-9bc3-d35be43cb707'} >2018-08-22 06:31:46.347 39124 INFO cinder.volume.flows.manager.create_volume [req-ae4437f2-14fd-4e66-9a8e-b20f45eac27e b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] Volume volume-fb72c65a-42ae-439d-9bc3-d35be43cb707 (fb72c65a-42ae-439d-9bc3-d35be43cb707): created successfully >2018-08-22 06:31:46.351 39124 INFO cinder.volume.manager [req-ae4437f2-14fd-4e66-9a8e-b20f45eac27e b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] Created volume successfully. >2018-08-22 06:31:47.896 39124 INFO cinder.volume.manager [req-8409d324-4f21-4f1b-8b11-efaf8936b89f b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] Retype volume completed successfully. >2018-08-22 06:31:52.519 39124 WARNING cinder.quota [req-ed5079dc-582c-4a99-a9f7-6cbba8628653 b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-626315363 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-626315363, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:31:52.520 39124 WARNING cinder.quota [req-ed5079dc-582c-4a99-a9f7-6cbba8628653 b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-626315363 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-626315363, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:31:52.772 39124 INFO cinder.volume.manager [req-ed5079dc-582c-4a99-a9f7-6cbba8628653 b91d349f50e34689bfca8ad4ad673c8b 6bfadccda38c49aa99a94e415729297a - default default] Deleted volume successfully. >2018-08-22 06:32:34.904 39124 INFO cinder.volume.flows.manager.create_volume [req-ab2dbae1-50f0-4710-940f-6f322b7dbd9e 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume 725d7be1-e90d-4e1e-bb5c-53a7a8dd87c8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-725d7be1-e90d-4e1e-bb5c-53a7a8dd87c8'} >2018-08-22 06:32:35.434 39124 INFO cinder.volume.flows.manager.create_volume [req-ab2dbae1-50f0-4710-940f-6f322b7dbd9e 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume volume-725d7be1-e90d-4e1e-bb5c-53a7a8dd87c8 (725d7be1-e90d-4e1e-bb5c-53a7a8dd87c8): created successfully >2018-08-22 06:32:35.437 39124 INFO cinder.volume.manager [req-ab2dbae1-50f0-4710-940f-6f322b7dbd9e 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Created volume successfully. >2018-08-22 06:32:36.415 39124 INFO cinder.volume.manager [req-6de93bd3-c519-4808-8fe2-b2ee6809afbb 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Attaching volume 725d7be1-e90d-4e1e-bb5c-53a7a8dd87c8 to instance 3b75c8d5-2fa5-4a23-9e2d-423a64f3ec30 at mountpoint /dev/vdb on host None. >2018-08-22 06:32:36.485 39124 INFO cinder.volume.manager [req-6de93bd3-c519-4808-8fe2-b2ee6809afbb 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Attach volume completed successfully. >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager [req-93ef5889-e912-4ad7-aed2-a7731f4bb0a4 19712bdc593a4c3289196b8fdb54a879 099d1fae187c4c63bd194e22ac84079c - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager force=force) >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-22 06:32:37.687 39124 ERROR cinder.volume.manager >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server [req-93ef5889-e912-4ad7-aed2-a7731f4bb0a4 19712bdc593a4c3289196b8fdb54a879 099d1fae187c4c63bd194e22ac84079c - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 06:32:37.689 39124 ERROR oslo_messaging.rpc.server >2018-08-22 06:32:38.006 39124 INFO cinder.volume.manager [req-65556b7a-0fe2-4daa-a041-a9c32641561d 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Detaching volume 725d7be1-e90d-4e1e-bb5c-53a7a8dd87c8 from instance 3b75c8d5-2fa5-4a23-9e2d-423a64f3ec30. >2018-08-22 06:32:38.091 39124 INFO cinder.volume.manager [req-65556b7a-0fe2-4daa-a041-a9c32641561d 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Detach volume completed successfully. >2018-08-22 06:32:40.409 39116 INFO cinder.volume.flows.manager.create_volume [req-0b26c505-369b-436b-847c-4ea6ee12af50 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume 34aaf858-62e3-41a3-801a-dba748e77ba0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-34aaf858-62e3-41a3-801a-dba748e77ba0'} >2018-08-22 06:32:40.946 39116 INFO cinder.volume.flows.manager.create_volume [req-0b26c505-369b-436b-847c-4ea6ee12af50 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume volume-34aaf858-62e3-41a3-801a-dba748e77ba0 (34aaf858-62e3-41a3-801a-dba748e77ba0): created successfully >2018-08-22 06:32:40.950 39116 INFO cinder.volume.manager [req-0b26c505-369b-436b-847c-4ea6ee12af50 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Created volume successfully. >2018-08-22 06:32:43.083 39116 INFO cinder.volume.manager [req-19e23729-5e05-4a17-b47f-4154991f3836 19712bdc593a4c3289196b8fdb54a879 099d1fae187c4c63bd194e22ac84079c - default default] Deleted volume successfully. >2018-08-22 06:32:43.621 39124 INFO cinder.volume.flows.manager.create_volume [req-b1bcef24-09d7-4c43-be36-99c0f177ed38 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume dd6b5dd9-6a39-423d-9792-7faf7efc9f2d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-dd6b5dd9-6a39-423d-9792-7faf7efc9f2d'} >2018-08-22 06:32:44.163 39124 INFO cinder.volume.flows.manager.create_volume [req-b1bcef24-09d7-4c43-be36-99c0f177ed38 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume volume-dd6b5dd9-6a39-423d-9792-7faf7efc9f2d (dd6b5dd9-6a39-423d-9792-7faf7efc9f2d): created successfully >2018-08-22 06:32:44.166 39124 INFO cinder.volume.manager [req-b1bcef24-09d7-4c43-be36-99c0f177ed38 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Created volume successfully. >2018-08-22 06:32:46.630 39124 INFO cinder.volume.manager [req-97b0f207-5a46-4120-8ff2-5ddb22511f87 19712bdc593a4c3289196b8fdb54a879 099d1fae187c4c63bd194e22ac84079c - default default] Deleted volume successfully. >2018-08-22 06:32:47.129 39124 INFO cinder.volume.flows.manager.create_volume [req-52af652a-d8bb-48d0-81f7-57ff7baf6cf1 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume d58d7e77-1b7a-4092-9faf-cb20edda5288: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d58d7e77-1b7a-4092-9faf-cb20edda5288'} >2018-08-22 06:32:47.657 39124 INFO cinder.volume.flows.manager.create_volume [req-52af652a-d8bb-48d0-81f7-57ff7baf6cf1 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume volume-d58d7e77-1b7a-4092-9faf-cb20edda5288 (d58d7e77-1b7a-4092-9faf-cb20edda5288): created successfully >2018-08-22 06:32:47.661 39124 INFO cinder.volume.manager [req-52af652a-d8bb-48d0-81f7-57ff7baf6cf1 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Created volume successfully. >2018-08-22 06:32:49.662 39124 INFO cinder.volume.manager [req-c6b7f61a-91dc-4689-8f7b-c1918604cb67 19712bdc593a4c3289196b8fdb54a879 099d1fae187c4c63bd194e22ac84079c - default default] Deleted volume successfully. >2018-08-22 06:32:49.994 39124 INFO cinder.volume.flows.manager.create_volume [req-ac419b06-529b-4cc7-b405-c3afb25feca4 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume 1cc4b1e5-5ea7-4b7b-aa2a-738a4245613b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1cc4b1e5-5ea7-4b7b-aa2a-738a4245613b'} >2018-08-22 06:32:50.519 39124 INFO cinder.volume.flows.manager.create_volume [req-ac419b06-529b-4cc7-b405-c3afb25feca4 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume volume-1cc4b1e5-5ea7-4b7b-aa2a-738a4245613b (1cc4b1e5-5ea7-4b7b-aa2a-738a4245613b): created successfully >2018-08-22 06:32:50.522 39124 INFO cinder.volume.manager [req-ac419b06-529b-4cc7-b405-c3afb25feca4 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Created volume successfully. >2018-08-22 06:32:52.492 39124 INFO cinder.volume.manager [req-ee6340eb-a129-4063-8c07-13def9df20ea 19712bdc593a4c3289196b8fdb54a879 099d1fae187c4c63bd194e22ac84079c - default default] Deleted volume successfully. >2018-08-22 06:32:52.939 39124 INFO cinder.volume.flows.manager.create_volume [req-b4809e65-3f02-403d-9d46-0459f03aa6ac 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume a2def912-0df3-4517-a6e5-8f0c12e78f7d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a2def912-0df3-4517-a6e5-8f0c12e78f7d'} >2018-08-22 06:32:53.662 39124 INFO cinder.volume.flows.manager.create_volume [req-b4809e65-3f02-403d-9d46-0459f03aa6ac 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Volume volume-a2def912-0df3-4517-a6e5-8f0c12e78f7d (a2def912-0df3-4517-a6e5-8f0c12e78f7d): created successfully >2018-08-22 06:32:53.665 39124 INFO cinder.volume.manager [req-b4809e65-3f02-403d-9d46-0459f03aa6ac 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Created volume successfully. >2018-08-22 06:32:56.817 39124 INFO cinder.volume.manager [req-7e689728-2f0f-4363-b039-2a40b8c28ee0 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Deleted volume successfully. >2018-08-22 06:32:57.103 39124 INFO cinder.volume.manager [req-0ee40366-f4b7-4597-9c80-61a57041ea09 5f79a01d7c1542ebafcf7283cd078497 c84089e815fb4b9aa63526c92d647685 - default default] Deleted volume successfully. >2018-08-22 06:33:04.896 39124 INFO cinder.volume.flows.manager.create_volume [req-8ba97cb8-f102-4d90-8165-8fa132c6339b 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Volume f6d0068d-9395-4a92-9417-01726f5f6c75: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f6d0068d-9395-4a92-9417-01726f5f6c75'} >2018-08-22 06:33:05.425 39124 INFO cinder.volume.flows.manager.create_volume [req-8ba97cb8-f102-4d90-8165-8fa132c6339b 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Volume volume-f6d0068d-9395-4a92-9417-01726f5f6c75 (f6d0068d-9395-4a92-9417-01726f5f6c75): created successfully >2018-08-22 06:33:05.429 39124 INFO cinder.volume.manager [req-8ba97cb8-f102-4d90-8165-8fa132c6339b 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Created volume successfully. >2018-08-22 06:33:07.117 39124 INFO cinder.volume.flows.manager.create_volume [req-a7fe7239-149a-440f-b0bd-b1c97c002168 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Volume 5f887757-4fa2-40a1-ad57-be342d364cf0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5f887757-4fa2-40a1-ad57-be342d364cf0'} >2018-08-22 06:33:07.645 39124 INFO cinder.volume.flows.manager.create_volume [req-a7fe7239-149a-440f-b0bd-b1c97c002168 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Volume volume-5f887757-4fa2-40a1-ad57-be342d364cf0 (5f887757-4fa2-40a1-ad57-be342d364cf0): created successfully >2018-08-22 06:33:07.649 39124 INFO cinder.volume.manager [req-a7fe7239-149a-440f-b0bd-b1c97c002168 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Created volume successfully. >2018-08-22 06:33:08.907 39116 INFO cinder.volume.flows.manager.create_volume [req-bef8de27-9242-4f71-aaaa-07681d1bad3a 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Volume b60203d2-889d-4a15-a05e-bb401609ee50: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b60203d2-889d-4a15-a05e-bb401609ee50'} >2018-08-22 06:33:09.533 39116 INFO cinder.volume.flows.manager.create_volume [req-bef8de27-9242-4f71-aaaa-07681d1bad3a 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Volume volume-b60203d2-889d-4a15-a05e-bb401609ee50 (b60203d2-889d-4a15-a05e-bb401609ee50): created successfully >2018-08-22 06:33:09.536 39116 INFO cinder.volume.manager [req-bef8de27-9242-4f71-aaaa-07681d1bad3a 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Created volume successfully. >2018-08-22 06:33:10.751 39124 INFO cinder.volume.flows.manager.create_volume [req-e9dc9efa-f022-497a-b4f5-5f0a9655139d 0da11f63f94a4ccca34525f1f31fa7de c510358804d4463ba874758d5796ab43 - default default] Volume 76628bda-27da-421b-95fd-3ca66442d785: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-76628bda-27da-421b-95fd-3ca66442d785'} >2018-08-22 06:33:11.367 39124 INFO cinder.volume.flows.manager.create_volume [req-e9dc9efa-f022-497a-b4f5-5f0a9655139d 0da11f63f94a4ccca34525f1f31fa7de c510358804d4463ba874758d5796ab43 - default default] Volume volume-76628bda-27da-421b-95fd-3ca66442d785 (76628bda-27da-421b-95fd-3ca66442d785): created successfully >2018-08-22 06:33:11.370 39124 INFO cinder.volume.manager [req-e9dc9efa-f022-497a-b4f5-5f0a9655139d 0da11f63f94a4ccca34525f1f31fa7de c510358804d4463ba874758d5796ab43 - default default] Created volume successfully. >2018-08-22 06:33:16.242 39124 INFO cinder.volume.manager [req-2a90b29d-8799-4b25-8cfd-1ded1e751971 0da11f63f94a4ccca34525f1f31fa7de c510358804d4463ba874758d5796ab43 - default default] Deleted volume successfully. >2018-08-22 06:33:16.274 39124 INFO cinder.volume.manager [req-8fcfde6f-3ef8-4ecf-9d44-57f9cff3a7d2 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Deleted volume successfully. >2018-08-22 06:33:16.383 39124 INFO cinder.volume.manager [req-d8d6bb13-6f31-445e-9cfc-27ee248947ed 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Deleted volume successfully. >2018-08-22 06:33:16.560 39116 INFO cinder.volume.manager [req-66d66a81-c5db-4f2e-a622-7f74829f34bd 507355b44496449fb422566782b9beb9 9924cffe22ba47139f34bc5f2a2c53d2 - default default] Deleted volume successfully. >2018-08-22 06:33:22.787 39124 INFO cinder.volume.flows.manager.create_volume [req-af01a5a4-b59d-4a26-8f4d-a4ac770d82f9 83979037bfb84884919fe104528045ba 4bafc8976ded4470b4c30544f467bbc5 - default default] Volume 6b981cf9-df77-40ec-b46d-f101578d2fba: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6b981cf9-df77-40ec-b46d-f101578d2fba'} >2018-08-22 06:33:23.318 39124 INFO cinder.volume.flows.manager.create_volume [req-af01a5a4-b59d-4a26-8f4d-a4ac770d82f9 83979037bfb84884919fe104528045ba 4bafc8976ded4470b4c30544f467bbc5 - default default] Volume volume-6b981cf9-df77-40ec-b46d-f101578d2fba (6b981cf9-df77-40ec-b46d-f101578d2fba): created successfully >2018-08-22 06:33:23.322 39124 INFO cinder.volume.manager [req-af01a5a4-b59d-4a26-8f4d-a4ac770d82f9 83979037bfb84884919fe104528045ba 4bafc8976ded4470b4c30544f467bbc5 - default default] Created volume successfully. >2018-08-22 06:33:30.890 39124 INFO cinder.volume.manager [req-9997b37b-7a5e-4913-9b4c-0cd94c614ddb 83979037bfb84884919fe104528045ba 4bafc8976ded4470b4c30544f467bbc5 - default default] Deleted volume successfully. >2018-08-22 06:33:38.171 39124 INFO cinder.volume.flows.manager.create_volume [req-1880b15e-50f4-4baa-8bdf-2a97a7a00e13 e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] Volume 3976a937-b62f-42c0-a5ab-c3b3eb2ed415: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3976a937-b62f-42c0-a5ab-c3b3eb2ed415'} >2018-08-22 06:33:38.715 39124 INFO cinder.volume.flows.manager.create_volume [req-1880b15e-50f4-4baa-8bdf-2a97a7a00e13 e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] Volume volume-3976a937-b62f-42c0-a5ab-c3b3eb2ed415 (3976a937-b62f-42c0-a5ab-c3b3eb2ed415): created successfully >2018-08-22 06:33:38.720 39124 INFO cinder.volume.manager [req-1880b15e-50f4-4baa-8bdf-2a97a7a00e13 e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] Created volume successfully. >2018-08-22 06:33:40.812 39124 INFO cinder.volume.manager [req-8776b6d9-ffdb-4d3e-8eac-ffc02357d35e 42601a1cf1864227b99e482e15679ce5 ef7c9ef4dffc41ac95aebbcb4ae9f347 - default default] Transfer volume completed successfully. >2018-08-22 06:33:43.072 39124 INFO cinder.volume.manager [req-b9d92c8f-6b35-4635-afd7-6048f04b9504 09cea8eafde44bc58238339a6e896fa6 03ca283692044949aa8ee9913335132f - default default] Deleted volume successfully. >2018-08-22 06:33:43.671 39124 INFO cinder.volume.flows.manager.create_volume [req-00170da5-6b90-43fb-944c-5765fb7bd57d e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] Volume 98f56b7a-3eda-43ae-b567-30fb3928b337: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-98f56b7a-3eda-43ae-b567-30fb3928b337'} >2018-08-22 06:33:44.196 39124 INFO cinder.volume.flows.manager.create_volume [req-00170da5-6b90-43fb-944c-5765fb7bd57d e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] Volume volume-98f56b7a-3eda-43ae-b567-30fb3928b337 (98f56b7a-3eda-43ae-b567-30fb3928b337): created successfully >2018-08-22 06:33:44.200 39124 INFO cinder.volume.manager [req-00170da5-6b90-43fb-944c-5765fb7bd57d e91d357e404f436488f3dbc9f4d587db ecd3d2497f3a4eb286ec574f9477cd50 - default default] Created volume successfully. >2018-08-22 06:33:47.062 39124 INFO cinder.volume.manager [req-6ca52f2b-8714-4f94-af58-23ea08f539d8 09cea8eafde44bc58238339a6e896fa6 03ca283692044949aa8ee9913335132f - default default] Deleted volume successfully. >2018-08-22 06:33:53.936 39124 INFO cinder.volume.flows.manager.create_volume [req-26f3a19d-dd2e-4d87-8c92-0ec552618e74 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Volume 1927250e-3575-47c8-9f38-507b6e99fdd7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1927250e-3575-47c8-9f38-507b6e99fdd7'} >2018-08-22 06:33:54.554 39124 INFO cinder.volume.flows.manager.create_volume [req-26f3a19d-dd2e-4d87-8c92-0ec552618e74 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Volume volume-1927250e-3575-47c8-9f38-507b6e99fdd7 (1927250e-3575-47c8-9f38-507b6e99fdd7): created successfully >2018-08-22 06:33:54.558 39124 INFO cinder.volume.manager [req-26f3a19d-dd2e-4d87-8c92-0ec552618e74 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Created volume successfully. >2018-08-22 06:34:00.618 39124 INFO cinder.volume.manager [req-1536c3a3-2f2d-4d1b-84f7-fcf5ded98834 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Attaching volume 1927250e-3575-47c8-9f38-507b6e99fdd7 to instance e12ed372-bd01-45d7-92c5-5f6eda81faa6 at mountpoint /dev/vdb on host None. >2018-08-22 06:34:00.694 39124 INFO cinder.volume.manager [req-1536c3a3-2f2d-4d1b-84f7-fcf5ded98834 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Attach volume completed successfully. >2018-08-22 06:34:01.466 39124 INFO cinder.volume.manager [req-1280887d-ab15-4627-8ab0-d44b2bdd4783 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Detaching volume 1927250e-3575-47c8-9f38-507b6e99fdd7 from instance e12ed372-bd01-45d7-92c5-5f6eda81faa6. >2018-08-22 06:34:01.553 39124 INFO cinder.volume.manager [req-1280887d-ab15-4627-8ab0-d44b2bdd4783 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Detach volume completed successfully. >2018-08-22 06:34:07.624 39124 INFO cinder.volume.manager [req-ceb45038-49af-4475-bf37-1ba0182e669d 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Attaching volume 1927250e-3575-47c8-9f38-507b6e99fdd7 to instance a5d7ea82-157d-46ec-8eb0-bec590da0e57 at mountpoint /dev/vdb on host None. >2018-08-22 06:34:07.694 39124 INFO cinder.volume.manager [req-ceb45038-49af-4475-bf37-1ba0182e669d 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Attach volume completed successfully. >2018-08-22 06:34:08.417 39124 INFO cinder.volume.manager [req-cc340b62-df7e-430b-bdbb-69108a1878e0 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Detaching volume 1927250e-3575-47c8-9f38-507b6e99fdd7 from instance a5d7ea82-157d-46ec-8eb0-bec590da0e57. >2018-08-22 06:34:08.502 39124 INFO cinder.volume.manager [req-cc340b62-df7e-430b-bdbb-69108a1878e0 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Detach volume completed successfully. >2018-08-22 06:34:21.655 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 06:34:39.513 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-71d7b48b-590b-411e-a81c-7c08f9b96e85 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:34:40.129 39124 INFO cinder.volume.manager [req-71d7b48b-590b-411e-a81c-7c08f9b96e85 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Copy volume to image completed successfully. >2018-08-22 06:34:43.712 39124 INFO cinder.volume.manager [req-778ebcf0-2bc5-4e07-a7c1-45a2a46a1ff1 3d54010dbdc94b79bc98c3c2b189cadb d40de1b30522489395271a101cdb5af5 - default default] Deleted volume successfully. >2018-08-22 06:34:50.181 39124 INFO cinder.volume.flows.manager.create_volume [req-448a53d5-fbcf-4acc-9d2e-bcd1a5808c68 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Volume d8b34e3f-a321-41f3-8a6e-9ffe6f221145: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d8b34e3f-a321-41f3-8a6e-9ffe6f221145'} >2018-08-22 06:34:51.075 39124 INFO cinder.volume.flows.manager.create_volume [req-448a53d5-fbcf-4acc-9d2e-bcd1a5808c68 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Volume volume-d8b34e3f-a321-41f3-8a6e-9ffe6f221145 (d8b34e3f-a321-41f3-8a6e-9ffe6f221145): created successfully >2018-08-22 06:34:51.079 39124 INFO cinder.volume.manager [req-448a53d5-fbcf-4acc-9d2e-bcd1a5808c68 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Created volume successfully. >2018-08-22 06:35:09.116 39124 INFO cinder.volume.manager [req-33f08ea7-598c-4066-bf9a-1d2baa418e73 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Initialize volume connection completed successfully. >2018-08-22 06:35:14.008 39124 INFO cinder.volume.manager [req-cda4a09b-ea82-4277-89a5-f83c083074ad c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Attaching volume d8b34e3f-a321-41f3-8a6e-9ffe6f221145 to instance 508bb28f-7dae-4102-856a-95bbaceb10e7 at mountpoint /dev/vdb on host None. >2018-08-22 06:35:14.086 39124 INFO cinder.volume.manager [req-cda4a09b-ea82-4277-89a5-f83c083074ad c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Attach volume completed successfully. >2018-08-22 06:35:14.869 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-54c7dca5-fd58-436c-b448-af125838c163 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Unable to get remote copy information for volume volume-d8b34e3f-a321-41f3-8a6e-9ffe6f221145 >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova [req-54c7dca5-fd58-436c-b448-af125838c163 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Failed to notify nova on events: [{'tag': 'd8b34e3f-a321-41f3-8a6e-9ffe6f221145', 'name': 'volume-extended', 'server_uuid': '508bb28f-7dae-4102-856a-95bbaceb10e7'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-f19ff59a-b3b6-44d6-ae37-7247aeb11abe) >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova return_raw=True) >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-f19ff59a-b3b6-44d6-ae37-7247aeb11abe) >2018-08-22 06:35:15.604 39124 ERROR cinder.compute.nova >2018-08-22 06:35:15.628 39124 INFO cinder.volume.manager [req-54c7dca5-fd58-436c-b448-af125838c163 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Extend volume completed successfully. >2018-08-22 06:40:31.185 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-368f7ee8-87e3-4c24-8748-206ccb45b918 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:40:31.581 39124 INFO cinder.volume.manager [req-368f7ee8-87e3-4c24-8748-206ccb45b918 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Terminate volume connection completed successfully. >2018-08-22 06:40:31.949 39124 INFO cinder.volume.manager [req-d8e924a8-f9aa-4239-8884-8f336766fc6b c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Detaching volume d8b34e3f-a321-41f3-8a6e-9ffe6f221145 from instance 508bb28f-7dae-4102-856a-95bbaceb10e7. >2018-08-22 06:40:32.034 39124 INFO cinder.volume.manager [req-d8e924a8-f9aa-4239-8884-8f336766fc6b c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Detach volume completed successfully. >2018-08-22 06:40:35.773 39124 INFO cinder.volume.manager [req-3fa256e1-f59a-4057-a8c5-968da1cd3680 c2f12005ce49455fb47abd0bffdd0f36 1ba44b9d7d5e4dee9acb89b570d6c384 - default default] Deleted volume successfully. >2018-08-22 06:40:41.304 39124 INFO cinder.volume.flows.manager.create_volume [req-47d9ab62-d8d6-48f2-a3e1-6d876d3d142d 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] Volume a866730f-6e2c-40b5-a70c-2c8b63d271f8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a866730f-6e2c-40b5-a70c-2c8b63d271f8'} >2018-08-22 06:40:41.887 39124 INFO cinder.volume.flows.manager.create_volume [req-47d9ab62-d8d6-48f2-a3e1-6d876d3d142d 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] Volume volume-a866730f-6e2c-40b5-a70c-2c8b63d271f8 (a866730f-6e2c-40b5-a70c-2c8b63d271f8): created successfully >2018-08-22 06:40:41.890 39124 INFO cinder.volume.manager [req-47d9ab62-d8d6-48f2-a3e1-6d876d3d142d 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] Created volume successfully. >2018-08-22 06:40:43.535 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-2851e299-37c9-4ab5-9d7c-e5f4e167983a 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] Unable to get remote copy information for volume volume-a866730f-6e2c-40b5-a70c-2c8b63d271f8 >2018-08-22 06:40:43.877 39124 INFO cinder.volume.manager [req-2851e299-37c9-4ab5-9d7c-e5f4e167983a 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] Extend volume completed successfully. >2018-08-22 06:40:46.298 39124 INFO cinder.volume.manager [req-e70d3919-2f3f-4f23-a4b0-cb9cff64a91f 7094846dcbf14163835b60b9bb03a0b5 d5da06081240442fb597877bf64705e2 - default default] Deleted volume successfully. >2018-08-22 06:40:50.598 39124 INFO cinder.volume.flows.manager.create_volume [req-281786d1-dae3-493d-a013-bb3c2cf09570 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume 45b1548a-00a2-401c-aeb7-7377c00928d6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-45b1548a-00a2-401c-aeb7-7377c00928d6'} >2018-08-22 06:40:51.134 39124 INFO cinder.volume.flows.manager.create_volume [req-281786d1-dae3-493d-a013-bb3c2cf09570 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume volume-45b1548a-00a2-401c-aeb7-7377c00928d6 (45b1548a-00a2-401c-aeb7-7377c00928d6): created successfully >2018-08-22 06:40:51.138 39124 INFO cinder.volume.manager [req-281786d1-dae3-493d-a013-bb3c2cf09570 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Created volume successfully. >2018-08-22 06:40:53.497 39124 INFO cinder.volume.flows.manager.create_volume [req-1fa88dec-8925-451e-aade-f3c7fa5a27c3 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume 798d34a6-c0ea-47ab-968a-2b5dd050eaa4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-798d34a6-c0ea-47ab-968a-2b5dd050eaa4'} >2018-08-22 06:40:54.027 39124 INFO cinder.volume.flows.manager.create_volume [req-1fa88dec-8925-451e-aade-f3c7fa5a27c3 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume volume-798d34a6-c0ea-47ab-968a-2b5dd050eaa4 (798d34a6-c0ea-47ab-968a-2b5dd050eaa4): created successfully >2018-08-22 06:40:54.031 39124 INFO cinder.volume.manager [req-1fa88dec-8925-451e-aade-f3c7fa5a27c3 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Created volume successfully. >2018-08-22 06:40:55.997 39124 INFO cinder.volume.manager [req-94b364c8-7168-436c-804c-da2b7541530c acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Deleted volume successfully. >2018-08-22 06:40:57.348 39124 INFO cinder.volume.manager [req-914e6546-f5eb-47ff-91f0-87533b337650 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Deleted volume successfully. >2018-08-22 06:40:57.753 39124 INFO cinder.volume.flows.manager.create_volume [req-ad707f2a-2985-4c63-9fd7-38956b08aa0f acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume f983e0b5-a951-4240-aa36-1228ef62c9a5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f983e0b5-a951-4240-aa36-1228ef62c9a5'} >2018-08-22 06:40:58.283 39124 INFO cinder.volume.flows.manager.create_volume [req-ad707f2a-2985-4c63-9fd7-38956b08aa0f acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume volume-f983e0b5-a951-4240-aa36-1228ef62c9a5 (f983e0b5-a951-4240-aa36-1228ef62c9a5): created successfully >2018-08-22 06:40:58.288 39124 INFO cinder.volume.manager [req-ad707f2a-2985-4c63-9fd7-38956b08aa0f acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Created volume successfully. >2018-08-22 06:40:59.214 39124 INFO cinder.volume.flows.manager.create_volume [req-8f6565a0-53c9-4be4-af5c-b0e732b07d8b acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume 35b84250-b875-4378-8db9-0cd2ca519e6f: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-35b84250-b875-4378-8db9-0cd2ca519e6f', 'source_volid': 'f983e0b5-a951-4240-aa36-1228ef62c9a5', 'source_volstatus': u'available'} >2018-08-22 06:41:00.696 39124 INFO cinder.volume.flows.manager.create_volume [req-8f6565a0-53c9-4be4-af5c-b0e732b07d8b acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume volume-35b84250-b875-4378-8db9-0cd2ca519e6f (35b84250-b875-4378-8db9-0cd2ca519e6f): created successfully >2018-08-22 06:41:00.700 39124 INFO cinder.volume.manager [req-8f6565a0-53c9-4be4-af5c-b0e732b07d8b acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Created volume successfully. >2018-08-22 06:41:02.223 39124 INFO cinder.volume.flows.manager.create_volume [req-75adbce0-ff11-4cf2-92f5-01e56a22ae59 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume 5ae9f5bf-0f4e-4ff5-9b62-d8ee8f08143a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5ae9f5bf-0f4e-4ff5-9b62-d8ee8f08143a'} >2018-08-22 06:41:02.792 39124 INFO cinder.volume.flows.manager.create_volume [req-75adbce0-ff11-4cf2-92f5-01e56a22ae59 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume volume-5ae9f5bf-0f4e-4ff5-9b62-d8ee8f08143a (5ae9f5bf-0f4e-4ff5-9b62-d8ee8f08143a): created successfully >2018-08-22 06:41:02.796 39124 INFO cinder.volume.manager [req-75adbce0-ff11-4cf2-92f5-01e56a22ae59 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Created volume successfully. >2018-08-22 06:41:04.525 39124 INFO cinder.volume.manager [req-e5852dfe-1c4a-4a03-a076-cb462b001475 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Deleted volume successfully. >2018-08-22 06:41:08.684 39124 INFO cinder.volume.manager [req-2bedddeb-86a9-41f8-9685-965222b6677d acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Deleted volume successfully. >2018-08-22 06:41:11.168 39124 INFO cinder.volume.flows.manager.create_volume [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume 317f8143-1af5-4f43-a824-c256a056ebc9: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-317f8143-1af5-4f43-a824-c256a056ebc9', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b1ca8d10>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 06:41:12.450 39124 INFO cinder.image.image_utils [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Image download 12.00 MB at 9.37 MB/s >2018-08-22 06:41:21.748 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] initialize_connection: Did not find a preferred node for volume volume-317f8143-1af5-4f43-a824-c256a056ebc9. >2018-08-22 06:41:22.499 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 06:41:27.406 39124 INFO cinder.image.image_utils [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 06:41:29.246 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:41:29.594 39124 INFO cinder.volume.flows.manager.create_volume [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume volume-317f8143-1af5-4f43-a824-c256a056ebc9 (317f8143-1af5-4f43-a824-c256a056ebc9): created successfully >2018-08-22 06:41:29.598 39124 INFO cinder.volume.manager [req-c3be99c6-9ae7-493a-b7de-dedeb00d19de acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Created volume successfully. >2018-08-22 06:41:30.697 39124 INFO cinder.volume.flows.manager.create_volume [req-ade4c27d-5798-42fb-b863-c414a8361d7b acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume fc987486-88f5-470d-9dec-7398fbda497b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fc987486-88f5-470d-9dec-7398fbda497b'} >2018-08-22 06:41:31.240 39124 INFO cinder.volume.flows.manager.create_volume [req-ade4c27d-5798-42fb-b863-c414a8361d7b acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Volume volume-fc987486-88f5-470d-9dec-7398fbda497b (fc987486-88f5-470d-9dec-7398fbda497b): created successfully >2018-08-22 06:41:31.244 39124 INFO cinder.volume.manager [req-ade4c27d-5798-42fb-b863-c414a8361d7b acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Created volume successfully. >2018-08-22 06:41:32.971 39124 INFO cinder.volume.manager [req-5874f77e-3748-45f3-8c7c-92cb49b69f35 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Deleted volume successfully. >2018-08-22 06:41:34.308 39124 INFO cinder.volume.manager [req-e52abc58-ccdb-4701-adde-e7af4c8b3cce acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Deleted volume successfully. >2018-08-22 06:41:35.381 39124 INFO cinder.volume.manager [req-3a0f580d-9914-4713-8e64-304e4ac8fd57 acafa6fa59d04ab99363bfd9f56343cc 01b1e7a8a022479898f5403599cab326 - default default] Deleted volume successfully. >2018-08-22 06:41:44.720 39124 INFO cinder.volume.flows.manager.create_volume [req-0b251d8b-e640-4934-95db-f1087076b00f ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Volume 86126c91-48f0-489f-bfb2-340417626b67: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-86126c91-48f0-489f-bfb2-340417626b67'} >2018-08-22 06:41:45.247 39124 INFO cinder.volume.flows.manager.create_volume [req-0b251d8b-e640-4934-95db-f1087076b00f ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Volume volume-86126c91-48f0-489f-bfb2-340417626b67 (86126c91-48f0-489f-bfb2-340417626b67): created successfully >2018-08-22 06:41:45.251 39124 INFO cinder.volume.manager [req-0b251d8b-e640-4934-95db-f1087076b00f ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Created volume successfully. >2018-08-22 06:41:46.962 39124 INFO cinder.volume.flows.manager.create_volume [req-b17ce061-24ef-471c-a246-8f77b01c28b7 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Volume c60656ed-8963-4d03-8dcc-60f74da4beb9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c60656ed-8963-4d03-8dcc-60f74da4beb9'} >2018-08-22 06:41:47.502 39124 INFO cinder.volume.flows.manager.create_volume [req-b17ce061-24ef-471c-a246-8f77b01c28b7 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Volume volume-c60656ed-8963-4d03-8dcc-60f74da4beb9 (c60656ed-8963-4d03-8dcc-60f74da4beb9): created successfully >2018-08-22 06:41:47.506 39124 INFO cinder.volume.manager [req-b17ce061-24ef-471c-a246-8f77b01c28b7 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Created volume successfully. >2018-08-22 06:41:48.952 39124 INFO cinder.volume.flows.manager.create_volume [req-4adb6fff-1a53-4660-9d39-974ab2b46801 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Volume 68fba256-d6d7-4690-951a-068914a787af: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-68fba256-d6d7-4690-951a-068914a787af'} >2018-08-22 06:41:49.478 39124 INFO cinder.volume.flows.manager.create_volume [req-4adb6fff-1a53-4660-9d39-974ab2b46801 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Volume volume-68fba256-d6d7-4690-951a-068914a787af (68fba256-d6d7-4690-951a-068914a787af): created successfully >2018-08-22 06:41:49.482 39124 INFO cinder.volume.manager [req-4adb6fff-1a53-4660-9d39-974ab2b46801 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Created volume successfully. >2018-08-22 06:41:55.720 39124 INFO cinder.volume.manager [req-762593e4-3b4c-4fb5-ba1f-0947f3373188 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Deleted volume successfully. >2018-08-22 06:41:55.741 39124 INFO cinder.volume.manager [req-6e40b576-3f92-4bed-9f66-70bc465714e9 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Deleted volume successfully. >2018-08-22 06:41:55.991 39124 INFO cinder.volume.manager [req-05134ceb-ea30-4b31-8691-64a7a4e30e98 ea6ad9bf241142d2a78dbc99f6d782b8 73ab17b385874928bc9d67d085b9ec23 - default default] Deleted volume successfully. >2018-08-22 06:42:00.699 39124 INFO cinder.volume.flows.manager.create_volume [req-5fcfd694-1e83-4355-be3a-88194d205e91 b02ef15290c44de0808806d1bab31a0e fb938f12dbb844d5a091dbead9b3bd6c - default default] Volume af147629-0665-405e-8ce7-7a774a7e195d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-af147629-0665-405e-8ce7-7a774a7e195d'} >2018-08-22 06:42:01.240 39124 INFO cinder.volume.flows.manager.create_volume [req-5fcfd694-1e83-4355-be3a-88194d205e91 b02ef15290c44de0808806d1bab31a0e fb938f12dbb844d5a091dbead9b3bd6c - default default] Volume volume-af147629-0665-405e-8ce7-7a774a7e195d (af147629-0665-405e-8ce7-7a774a7e195d): created successfully >2018-08-22 06:42:01.245 39124 INFO cinder.volume.manager [req-5fcfd694-1e83-4355-be3a-88194d205e91 b02ef15290c44de0808806d1bab31a0e fb938f12dbb844d5a091dbead9b3bd6c - default default] Created volume successfully. >2018-08-22 06:42:23.360 39124 INFO cinder.volume.manager [req-efb40b52-82eb-41d5-9d62-32e47d632303 b02ef15290c44de0808806d1bab31a0e fb938f12dbb844d5a091dbead9b3bd6c - default default] Deleted volume successfully. >2018-08-22 06:42:27.976 39124 INFO cinder.volume.flows.manager.create_volume [req-fa5356ff-585e-41cc-a648-f6139b6a757b 8918f0e5e279464eb0519587aabb0c5e b0ee84d889714a05831d99865b7373af - default default] Volume beaff6a8-ec0c-4b90-89bc-7ac3de2fd688: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-beaff6a8-ec0c-4b90-89bc-7ac3de2fd688'} >2018-08-22 06:42:28.517 39124 INFO cinder.volume.flows.manager.create_volume [req-fa5356ff-585e-41cc-a648-f6139b6a757b 8918f0e5e279464eb0519587aabb0c5e b0ee84d889714a05831d99865b7373af - default default] Volume volume-beaff6a8-ec0c-4b90-89bc-7ac3de2fd688 (beaff6a8-ec0c-4b90-89bc-7ac3de2fd688): created successfully >2018-08-22 06:42:28.521 39124 INFO cinder.volume.manager [req-fa5356ff-585e-41cc-a648-f6139b6a757b 8918f0e5e279464eb0519587aabb0c5e b0ee84d889714a05831d99865b7373af - default default] Created volume successfully. >2018-08-22 06:42:31.096 39124 INFO cinder.volume.manager [req-32a43635-d042-4c37-838f-3fdfca5f57a2 8918f0e5e279464eb0519587aabb0c5e b0ee84d889714a05831d99865b7373af - default default] Deleted volume successfully. >2018-08-22 06:43:18.355 39124 INFO cinder.volume.flows.manager.create_volume [req-bf2cacdc-eae1-4df7-a9a0-a38bf227a357 bb7b938b82404e27b061243dbf35f948 2f684589186e4ea0ab65dd5a5622e740 - default default] Volume e812e572-07bb-4d79-a37f-4fad2e20836e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e812e572-07bb-4d79-a37f-4fad2e20836e'} >2018-08-22 06:43:18.900 39124 INFO cinder.volume.flows.manager.create_volume [req-bf2cacdc-eae1-4df7-a9a0-a38bf227a357 bb7b938b82404e27b061243dbf35f948 2f684589186e4ea0ab65dd5a5622e740 - default default] Volume volume-e812e572-07bb-4d79-a37f-4fad2e20836e (e812e572-07bb-4d79-a37f-4fad2e20836e): created successfully >2018-08-22 06:43:18.904 39124 INFO cinder.volume.manager [req-bf2cacdc-eae1-4df7-a9a0-a38bf227a357 bb7b938b82404e27b061243dbf35f948 2f684589186e4ea0ab65dd5a5622e740 - default default] Created volume successfully. >2018-08-22 06:43:22.587 39124 INFO cinder.volume.manager [req-8572ae61-261a-4928-b8b1-32576b0a2274 bb7b938b82404e27b061243dbf35f948 2f684589186e4ea0ab65dd5a5622e740 - default default] Deleted volume successfully. >2018-08-22 06:43:28.330 39124 INFO cinder.volume.flows.manager.create_volume [req-f8239f60-6e0a-4bcb-be39-0ab3614f773f c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Volume 60301e32-702e-4c48-90f4-dcddd00c96a8: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-60301e32-702e-4c48-90f4-dcddd00c96a8', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b242c2d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 06:43:29.145 39124 INFO cinder.image.image_utils [req-f8239f60-6e0a-4bcb-be39-0ab3614f773f c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 06:43:44.285 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 06:43:49.609 39124 INFO cinder.image.image_utils [req-f8239f60-6e0a-4bcb-be39-0ab3614f773f c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 06:43:52.342 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f8239f60-6e0a-4bcb-be39-0ab3614f773f c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:43:52.984 39124 INFO cinder.volume.flows.manager.create_volume [req-f8239f60-6e0a-4bcb-be39-0ab3614f773f c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Volume volume-60301e32-702e-4c48-90f4-dcddd00c96a8 (60301e32-702e-4c48-90f4-dcddd00c96a8): created successfully >2018-08-22 06:43:52.988 39124 INFO cinder.volume.manager [req-f8239f60-6e0a-4bcb-be39-0ab3614f773f c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Created volume successfully. >2018-08-22 06:43:54.680 39124 INFO cinder.volume.flows.manager.create_volume [req-c8f7620e-4cf0-4c99-a103-aa24c475d1ae c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Volume 58aad57d-7290-4eb2-b7dc-0056485b245e: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-58aad57d-7290-4eb2-b7dc-0056485b245e', 'source_volid': '60301e32-702e-4c48-90f4-dcddd00c96a8', 'source_volstatus': u'available'} >2018-08-22 06:43:57.451 39124 INFO cinder.volume.flows.manager.create_volume [req-c8f7620e-4cf0-4c99-a103-aa24c475d1ae c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Volume volume-58aad57d-7290-4eb2-b7dc-0056485b245e (58aad57d-7290-4eb2-b7dc-0056485b245e): created successfully >2018-08-22 06:43:57.455 39124 INFO cinder.volume.manager [req-c8f7620e-4cf0-4c99-a103-aa24c475d1ae c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Created volume successfully. >2018-08-22 06:44:02.489 39124 INFO cinder.volume.manager [req-3e8855c1-9b39-411e-b12b-bcb504d19a6a c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Deleted volume successfully. >2018-08-22 06:44:03.183 39124 INFO cinder.volume.manager [req-03241bee-c8d1-4676-9f75-32e9468e4abe c3f8b31e240248f78706a96c3edcdbad d21595ac27bf4a2f9214525235136e01 - default default] Deleted volume successfully. >2018-08-22 06:44:07.594 39124 INFO cinder.volume.flows.manager.create_volume [req-1daad89b-b978-4748-ad57-4492ee02b6d5 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Volume 0f8bb976-5e25-4a1a-9d46-00cb3129bea8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0f8bb976-5e25-4a1a-9d46-00cb3129bea8'} >2018-08-22 06:44:08.464 39124 INFO cinder.volume.flows.manager.create_volume [req-1daad89b-b978-4748-ad57-4492ee02b6d5 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Volume volume-0f8bb976-5e25-4a1a-9d46-00cb3129bea8 (0f8bb976-5e25-4a1a-9d46-00cb3129bea8): created successfully >2018-08-22 06:44:08.468 39124 INFO cinder.volume.manager [req-1daad89b-b978-4748-ad57-4492ee02b6d5 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Created volume successfully. >2018-08-22 06:44:09.795 39124 INFO cinder.volume.flows.manager.create_volume [req-901fac17-492e-4d51-941c-b08486e2fb03 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Volume 6812e72e-5cc1-4c8f-92e8-ef56342562c5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6812e72e-5cc1-4c8f-92e8-ef56342562c5'} >2018-08-22 06:44:10.657 39124 INFO cinder.volume.flows.manager.create_volume [req-901fac17-492e-4d51-941c-b08486e2fb03 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Volume volume-6812e72e-5cc1-4c8f-92e8-ef56342562c5 (6812e72e-5cc1-4c8f-92e8-ef56342562c5): created successfully >2018-08-22 06:44:10.661 39124 INFO cinder.volume.manager [req-901fac17-492e-4d51-941c-b08486e2fb03 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Created volume successfully. >2018-08-22 06:44:11.614 39116 INFO cinder.volume.flows.manager.create_volume [req-826877c1-8f4e-4c59-9a5c-5cb5bdb4985c 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Volume f7164b55-994a-49ac-8db4-f13a24b93594: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f7164b55-994a-49ac-8db4-f13a24b93594'} >2018-08-22 06:44:12.484 39116 INFO cinder.volume.flows.manager.create_volume [req-826877c1-8f4e-4c59-9a5c-5cb5bdb4985c 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Volume volume-f7164b55-994a-49ac-8db4-f13a24b93594 (f7164b55-994a-49ac-8db4-f13a24b93594): created successfully >2018-08-22 06:44:12.488 39116 INFO cinder.volume.manager [req-826877c1-8f4e-4c59-9a5c-5cb5bdb4985c 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Created volume successfully. >2018-08-22 06:44:16.481 39116 INFO cinder.volume.manager [req-a1937079-673e-47e1-bfb1-225fb5ca03bb 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Deleted volume successfully. >2018-08-22 06:44:16.501 39124 INFO cinder.volume.manager [req-59d4763e-bcfc-49e7-8ce8-2056b524f094 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Deleted volume successfully. >2018-08-22 06:44:16.703 39124 INFO cinder.volume.manager [req-0a7960cd-4fec-47b1-891e-a18cd020f482 6218f09b61134d43856f16f452d926ab 824ea84ff6fe4e159c0bee47d3bddf1f - default default] Deleted volume successfully. >2018-08-22 06:44:36.819 39116 INFO cinder.volume.flows.manager.create_volume [req-1f67d231-3f9c-4756-9874-0cb174526e3d 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Volume 43c83024-db0e-4491-89b9-82abd1e37778: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-43c83024-db0e-4491-89b9-82abd1e37778'} >2018-08-22 06:44:37.424 39116 INFO cinder.volume.flows.manager.create_volume [req-1f67d231-3f9c-4756-9874-0cb174526e3d 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Volume volume-43c83024-db0e-4491-89b9-82abd1e37778 (43c83024-db0e-4491-89b9-82abd1e37778): created successfully >2018-08-22 06:44:37.428 39116 INFO cinder.volume.manager [req-1f67d231-3f9c-4756-9874-0cb174526e3d 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Created volume successfully. >2018-08-22 06:44:39.116 39124 INFO cinder.volume.flows.manager.create_volume [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Volume b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3'} >2018-08-22 06:44:39.662 39124 INFO cinder.volume.flows.manager.create_volume [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Volume volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3 (b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3): created successfully >2018-08-22 06:44:39.666 39124 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Created volume successfully. >2018-08-22 06:44:46.790 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] initialize_connection: Did not find a preferred node for volume volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3. >2018-08-22 06:44:47.058 39124 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Initialize volume connection completed successfully. >2018-08-22 06:44:47.454 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 06:44:51.379 39116 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Initialize volume connection completed successfully. >2018-08-22 06:44:51.764 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 06:44:57.089 39116 INFO cinder.volume.utils [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Volume copy 1024.00 MB at 435.86 MB/s >2018-08-22 06:44:58.999 39124 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Terminate volume connection completed successfully. >2018-08-22 06:44:59.039 39124 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Remove volume export completed successfully. >2018-08-22 06:45:00.899 39116 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:45:01.175 39116 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Terminate volume connection completed successfully. >2018-08-22 06:45:01.207 39116 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Remove volume export completed successfully. >2018-08-22 06:45:01.455 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 06:45:06.007 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Error running SSH command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 06:45:06.009 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 06:45:06.009 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Error running SSH command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 06:45:06.010 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-43c83024-db0e-4491-89b9-82abd1e37778', '"volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-43c83024-db0e-4491-89b9-82abd1e37778 "volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 06:45:06.011 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Unable to rename the logical volume for volume: 43c83024-db0e-4491-89b9-82abd1e37778: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-43c83024-db0e-4491-89b9-82abd1e37778', '"volume-b75b6c0b-0686-4c9e-ba5d-6426c4f5efb3"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-22 06:45:06.160 39116 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Complete-Migrate volume completed successfully. >2018-08-22 06:45:06.161 39116 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Migrate volume completed successfully. >2018-08-22 06:45:06.678 39116 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Retype volume completed successfully. >2018-08-22 06:45:07.391 39116 INFO cinder.volume.manager [req-41d77e03-5283-417d-a7e6-be9bab9a340f 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Deleted volume successfully. >2018-08-22 06:45:08.554 39124 WARNING cinder.quota [req-05d19ae0-a496-426e-a898-deee52654139 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-470528033 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-470528033, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:45:08.555 39124 WARNING cinder.quota [req-05d19ae0-a496-426e-a898-deee52654139 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-470528033 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-470528033, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 06:45:08.811 39124 INFO cinder.volume.manager [req-05d19ae0-a496-426e-a898-deee52654139 31d7f1e0f39744bf9f81c8bb5baa5069 9bdcaf7d00ee4a1db32109470ff2219a - default default] Deleted volume successfully. >2018-08-22 06:45:29.324 39124 INFO cinder.volume.flows.manager.create_volume [req-074c9329-57d0-4741-9854-97a2bf5170c5 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Volume 3f512f38-d67d-48cb-8563-10695031732e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3f512f38-d67d-48cb-8563-10695031732e'} >2018-08-22 06:45:29.860 39124 INFO cinder.volume.flows.manager.create_volume [req-074c9329-57d0-4741-9854-97a2bf5170c5 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Volume volume-3f512f38-d67d-48cb-8563-10695031732e (3f512f38-d67d-48cb-8563-10695031732e): created successfully >2018-08-22 06:45:29.863 39124 INFO cinder.volume.manager [req-074c9329-57d0-4741-9854-97a2bf5170c5 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Created volume successfully. >2018-08-22 06:45:35.590 39124 INFO cinder.volume.manager [req-0394c428-cda5-4df8-9e5a-2fd07bdb4f8d f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Create snapshot completed successfully >2018-08-22 06:45:38.609 39124 INFO cinder.volume.manager [req-87ce0342-c4d4-4a11-86e5-c8c7da0ee105 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Create snapshot completed successfully >2018-08-22 06:45:41.903 39124 INFO cinder.volume.manager [req-53c95005-c532-4cfb-a574-65e2c3c2a378 ec213a9f7c54477f8a6327d04590b91f 7363a8784e174ee8a219a43b29eeaf20 - default default] Delete snapshot completed successfully. >2018-08-22 06:45:44.117 39124 INFO cinder.volume.manager [req-e4f44945-1023-45fa-a4e1-5ff27cfa5466 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Create snapshot completed successfully >2018-08-22 06:45:47.106 39124 INFO cinder.volume.manager [req-13bf053d-f4f9-43cb-96ba-7ad8a5203b0b ec213a9f7c54477f8a6327d04590b91f 7363a8784e174ee8a219a43b29eeaf20 - default default] Delete snapshot completed successfully. >2018-08-22 06:45:49.088 39124 INFO cinder.volume.manager [req-c6114ef2-909d-4e15-aa3c-bf877bed714e f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Create snapshot completed successfully >2018-08-22 06:45:52.160 39124 INFO cinder.volume.manager [req-d9eb6d3d-3695-4742-ad5c-958fa7344676 ec213a9f7c54477f8a6327d04590b91f 7363a8784e174ee8a219a43b29eeaf20 - default default] Delete snapshot completed successfully. >2018-08-22 06:45:54.454 39124 INFO cinder.volume.manager [req-61d54c25-e24b-45ff-8efa-40d9cbd06d70 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Create snapshot completed successfully >2018-08-22 06:45:57.678 39124 INFO cinder.volume.manager [req-b14ae8a9-0dc6-4e8e-921b-bc29cdbfb86f ec213a9f7c54477f8a6327d04590b91f 7363a8784e174ee8a219a43b29eeaf20 - default default] Delete snapshot completed successfully. >2018-08-22 06:46:01.504 39124 INFO cinder.volume.manager [req-cf01488d-d927-4d00-a11c-52faac270c20 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Delete snapshot completed successfully. >2018-08-22 06:46:03.287 39124 INFO cinder.volume.manager [req-d16c2e63-8e69-4dbc-93d2-db6cc5a9dce9 f4f5b7b5b56e4d0599feee14abf9f866 8a42351f4c6c4ac0a10159acb85cccbc - default default] Deleted volume successfully. >2018-08-22 06:46:08.546 39124 INFO cinder.volume.flows.manager.create_volume [req-7e479746-a763-4984-9b7a-5b0a1c88ab10 8c9eb61b14a04378b9ec1d7609328675 2346b43916994ced867d0331d16d0199 - default default] Volume e00cebb3-7953-4781-bbfb-2cdd87f24685: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e00cebb3-7953-4781-bbfb-2cdd87f24685'} >2018-08-22 06:46:09.091 39124 INFO cinder.volume.flows.manager.create_volume [req-7e479746-a763-4984-9b7a-5b0a1c88ab10 8c9eb61b14a04378b9ec1d7609328675 2346b43916994ced867d0331d16d0199 - default default] Volume volume-e00cebb3-7953-4781-bbfb-2cdd87f24685 (e00cebb3-7953-4781-bbfb-2cdd87f24685): created successfully >2018-08-22 06:46:09.095 39124 INFO cinder.volume.manager [req-7e479746-a763-4984-9b7a-5b0a1c88ab10 8c9eb61b14a04378b9ec1d7609328675 2346b43916994ced867d0331d16d0199 - default default] Created volume successfully. >2018-08-22 06:46:12.055 39124 INFO cinder.volume.manager [req-65890365-d4bf-4dfc-9b96-17163b998d22 8c9eb61b14a04378b9ec1d7609328675 2346b43916994ced867d0331d16d0199 - default default] Create snapshot completed successfully >2018-08-22 06:46:16.055 39124 INFO cinder.volume.manager [req-f119d1f6-62c2-4d36-a311-b55361dd85b4 8c9eb61b14a04378b9ec1d7609328675 2346b43916994ced867d0331d16d0199 - default default] Delete snapshot completed successfully. >2018-08-22 06:46:17.558 39124 INFO cinder.volume.manager [req-6da727dc-75a7-465a-b411-eea10329bf2b 8c9eb61b14a04378b9ec1d7609328675 2346b43916994ced867d0331d16d0199 - default default] Deleted volume successfully. >2018-08-22 06:46:21.893 39124 INFO cinder.volume.flows.manager.create_volume [req-b7d9e899-d9ac-4e03-910b-09284aca1c3e 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Volume b15b9b3e-3e7a-4581-9bab-f5a9116f50df: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b15b9b3e-3e7a-4581-9bab-f5a9116f50df'} >2018-08-22 06:46:22.424 39124 INFO cinder.volume.flows.manager.create_volume [req-b7d9e899-d9ac-4e03-910b-09284aca1c3e 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Volume volume-b15b9b3e-3e7a-4581-9bab-f5a9116f50df (b15b9b3e-3e7a-4581-9bab-f5a9116f50df): created successfully >2018-08-22 06:46:22.428 39124 INFO cinder.volume.manager [req-b7d9e899-d9ac-4e03-910b-09284aca1c3e 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Created volume successfully. >2018-08-22 06:46:25.183 39124 INFO cinder.volume.manager [req-df228169-2629-44ba-bb91-50d9b930f627 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:46:36.137 39124 INFO cinder.volume.manager [req-2a08a35a-3e29-4ad9-a42b-839ce681e58d 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Initialize volume connection completed successfully. >2018-08-22 06:47:00.261 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-2a08a35a-3e29-4ad9-a42b-839ce681e58d 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:47:00.539 39124 INFO cinder.volume.manager [req-2a08a35a-3e29-4ad9-a42b-839ce681e58d 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Terminate volume connection completed successfully. >2018-08-22 06:47:00.581 39124 INFO cinder.volume.manager [req-2a08a35a-3e29-4ad9-a42b-839ce681e58d 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Remove volume export completed successfully. >2018-08-22 06:47:02.005 39124 INFO cinder.volume.manager [req-2a08a35a-3e29-4ad9-a42b-839ce681e58d 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Deleted volume successfully. >2018-08-22 06:47:16.285 39124 INFO cinder.volume.manager [req-31c937c2-abab-4b86-8d5d-5dd68e7cc5e8 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Initialize volume connection completed successfully. >2018-08-22 06:47:20.839 39124 INFO cinder.volume.manager [req-99a36a6d-a586-42fb-9c01-5a619504b427 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Attaching volume b15b9b3e-3e7a-4581-9bab-f5a9116f50df to instance 0d40e602-24e8-4e3a-bd7b-55017c482aef at mountpoint /dev/vdb on host None. >2018-08-22 06:47:20.932 39124 INFO cinder.volume.manager [req-99a36a6d-a586-42fb-9c01-5a619504b427 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Attach volume completed successfully. >2018-08-22 06:47:25.323 39124 INFO cinder.volume.manager [req-280c122d-cf32-4260-bf5f-40dbeb5d152e 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:47:28.199 39124 INFO cinder.volume.manager [req-d1546b7c-0b77-494b-8a38-022938834bab 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:47:31.421 39124 INFO cinder.volume.manager [req-4c05e418-4255-4d16-bf32-463a31166a83 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:47:34.973 39124 INFO cinder.volume.manager [req-fd681653-32de-468f-9a90-993150b34a8f 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:47:38.180 39124 INFO cinder.volume.manager [req-d278c436-edcd-47c0-b5ba-80d0485dbec4 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:47:42.115 39124 INFO cinder.volume.manager [req-a71ebf1b-6dff-4f61-bac3-7c04992a9748 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:47:54.129 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-794d6423-6515-4337-86b1-614f5876a09d 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:47:54.580 39124 INFO cinder.volume.manager [req-794d6423-6515-4337-86b1-614f5876a09d 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Terminate volume connection completed successfully. >2018-08-22 06:47:54.708 39124 INFO cinder.volume.manager [req-04345666-8985-4801-80b8-8f265df75d35 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Detaching volume b15b9b3e-3e7a-4581-9bab-f5a9116f50df from instance 0d40e602-24e8-4e3a-bd7b-55017c482aef. >2018-08-22 06:47:54.796 39124 INFO cinder.volume.manager [req-04345666-8985-4801-80b8-8f265df75d35 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Detach volume completed successfully. >2018-08-22 06:48:00.008 39124 INFO cinder.volume.manager [req-c97a2544-59f0-41b2-ad70-607ce6f8c50c 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:48:03.698 39124 INFO cinder.volume.manager [req-ef15777f-fdf1-4fbc-8a26-c2d156ab0744 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:48:07.393 39124 INFO cinder.volume.manager [req-acb5791c-0ffe-4561-9617-59bb0bda92dc 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:48:24.739 39124 INFO cinder.volume.manager [req-9c39d0e5-1085-44bf-8c95-58a5e0b8ea05 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Initialize volume connection completed successfully. >2018-08-22 06:48:29.277 39124 INFO cinder.volume.manager [req-069c5dd1-64b4-4131-8e77-7164a6272e2a 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Attaching volume b15b9b3e-3e7a-4581-9bab-f5a9116f50df to instance 62183444-b6ff-4ac1-a179-7950e47b9c0c at mountpoint /dev/vdb on host None. >2018-08-22 06:48:29.348 39124 INFO cinder.volume.manager [req-069c5dd1-64b4-4131-8e77-7164a6272e2a 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Attach volume completed successfully. >2018-08-22 06:48:32.413 39124 INFO cinder.volume.manager [req-4eb463b9-6908-4f0e-83e0-35b78a49ff03 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:48:34.453 39124 INFO cinder.volume.manager [req-7f315dbf-8e45-4ea3-ba53-1e2921e2629a 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:48:37.154 39124 INFO cinder.volume.manager [req-fd5852e7-e5f2-4c3a-97d0-08597d744883 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:48:39.434 39124 INFO cinder.volume.manager [req-e712390c-a713-46a2-9467-ee66fb1eb893 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:48:41.678 39124 INFO cinder.volume.manager [req-d940b5d6-533a-47e7-9ef6-c8fda45d7efe 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:48:54.236 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b3cf6faf-fbcc-45b2-b1fe-8b3851486662 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:48:54.514 39124 INFO cinder.volume.manager [req-b3cf6faf-fbcc-45b2-b1fe-8b3851486662 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Terminate volume connection completed successfully. >2018-08-22 06:48:54.643 39124 INFO cinder.volume.manager [req-f44218ba-2492-4592-87b4-288c3632e186 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Detaching volume b15b9b3e-3e7a-4581-9bab-f5a9116f50df from instance 62183444-b6ff-4ac1-a179-7950e47b9c0c. >2018-08-22 06:48:54.730 39124 INFO cinder.volume.manager [req-f44218ba-2492-4592-87b4-288c3632e186 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Detach volume completed successfully. >2018-08-22 06:48:57.920 39124 INFO cinder.volume.flows.manager.create_volume [req-2c3c786a-7641-4234-8e9e-8e0a7c6ab0a9 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Volume 916b5bfe-6f85-4c77-ae80-7f172c03747c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-916b5bfe-6f85-4c77-ae80-7f172c03747c'} >2018-08-22 06:48:58.662 39124 INFO cinder.volume.flows.manager.create_volume [req-2c3c786a-7641-4234-8e9e-8e0a7c6ab0a9 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Volume volume-916b5bfe-6f85-4c77-ae80-7f172c03747c (916b5bfe-6f85-4c77-ae80-7f172c03747c): created successfully >2018-08-22 06:48:58.666 39124 INFO cinder.volume.manager [req-2c3c786a-7641-4234-8e9e-8e0a7c6ab0a9 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Created volume successfully. >2018-08-22 06:49:00.738 39124 INFO cinder.volume.manager [req-c0c42070-6b1c-4a2d-bfcf-22512b2adcec 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Create snapshot completed successfully >2018-08-22 06:49:01.830 39124 INFO cinder.volume.flows.manager.create_volume [req-3158e041-d819-4a34-99f9-d5aec7c15ba4 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Volume da2913c2-5531-4c5f-9439-1ace8ea34f14: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': 'f1d5fe08-e037-467e-afdb-301d2635397f', 'volume_name': 'volume-da2913c2-5531-4c5f-9439-1ace8ea34f14'} >2018-08-22 06:49:51.665 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3158e041-d819-4a34-99f9-d5aec7c15ba4 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Unable to get remote copy information for volume volume-da2913c2-5531-4c5f-9439-1ace8ea34f14 >2018-08-22 06:49:51.972 39124 INFO cinder.volume.flows.manager.create_volume [req-3158e041-d819-4a34-99f9-d5aec7c15ba4 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Volume volume-da2913c2-5531-4c5f-9439-1ace8ea34f14 (da2913c2-5531-4c5f-9439-1ace8ea34f14): created successfully >2018-08-22 06:49:51.976 39124 INFO cinder.volume.manager [req-3158e041-d819-4a34-99f9-d5aec7c15ba4 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Created volume successfully. >2018-08-22 06:49:54.529 39124 INFO cinder.volume.manager [req-5eb47b45-7154-449d-88f8-51095e1d0c8b 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Deleted volume successfully. >2018-08-22 06:49:58.670 39124 INFO cinder.volume.manager [req-283e23f8-478a-4227-b4b8-59e30977cd35 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:49:58.885 39124 INFO cinder.volume.manager [req-224d30b5-4cd3-4256-b6ac-3b6027c49936 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Delete snapshot completed successfully. >2018-08-22 06:50:01.325 39124 WARNING cinder.db.sqlalchemy.api [req-5c026852-3b02-4174-a8dc-9cf727ab7bf0 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-22 06:50:01.336 39124 INFO cinder.volume.manager [req-ab748b1b-9779-4521-9396-c16b18b4a06b 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Deleted volume successfully. >2018-08-22 06:50:01.579 39124 INFO cinder.volume.manager [req-5c026852-3b02-4174-a8dc-9cf727ab7bf0 430554d7c1774a6fbd939e8bae0e7df6 c69ebd88a9924110af41938bcf82bfce - default default] Deleted volume successfully. >2018-08-22 06:50:06.409 39124 INFO cinder.volume.flows.manager.create_volume [req-300a25f9-8714-4d4d-832c-fb2b48c56e91 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Volume 15fccd0a-2348-4633-b2af-b91b86d30717: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-15fccd0a-2348-4633-b2af-b91b86d30717'} >2018-08-22 06:50:06.939 39124 INFO cinder.volume.flows.manager.create_volume [req-300a25f9-8714-4d4d-832c-fb2b48c56e91 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Volume volume-15fccd0a-2348-4633-b2af-b91b86d30717 (15fccd0a-2348-4633-b2af-b91b86d30717): created successfully >2018-08-22 06:50:06.942 39124 INFO cinder.volume.manager [req-300a25f9-8714-4d4d-832c-fb2b48c56e91 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Created volume successfully. >2018-08-22 06:50:10.096 39124 INFO cinder.volume.manager [req-a757e012-a203-42dc-ac00-4cd3972d7aad 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Create snapshot completed successfully >2018-08-22 06:50:16.014 39124 INFO cinder.volume.manager [req-411072f5-428a-4fb7-90a8-39f23253f014 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Create snapshot completed successfully >2018-08-22 06:50:18.681 39124 INFO cinder.volume.manager [req-96402120-c898-4e89-87c8-ae340d6971cb 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Create snapshot completed successfully >2018-08-22 06:50:24.614 39124 INFO cinder.volume.manager [req-76d9e09b-1c77-4d5f-a9c3-ecc8a7865625 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Delete snapshot completed successfully. >2018-08-22 06:50:24.828 39124 INFO cinder.volume.manager [req-88bc3faf-3560-4982-bce2-e9ba63e691d2 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Delete snapshot completed successfully. >2018-08-22 06:50:25.047 39124 INFO cinder.volume.manager [req-8c56f7fd-935c-482c-8ef9-a08c72c0d8d8 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Delete snapshot completed successfully. >2018-08-22 06:50:27.289 39124 INFO cinder.volume.manager [req-65ed4d37-b113-4695-a8d6-9cf70b66ec9c 8f67dd75248641f28db00a348d99b694 567d7840cce5475fbc8ecfd74fa988cf - default default] Deleted volume successfully. >2018-08-22 06:50:32.692 39124 INFO cinder.volume.flows.manager.create_volume [req-d98136d4-39ff-4a16-99d3-ed2fcf8fe462 4f0c9e8fc02e4045b7a37d9b34213b47 656613c3c72f40a495ee8b404e8b7c3a - default default] Volume 2c44a5de-50ea-4fca-87e4-7bfa1531030b: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-2c44a5de-50ea-4fca-87e4-7bfa1531030b'} >2018-08-22 06:50:33.219 39124 INFO cinder.volume.flows.manager.create_volume [req-d98136d4-39ff-4a16-99d3-ed2fcf8fe462 4f0c9e8fc02e4045b7a37d9b34213b47 656613c3c72f40a495ee8b404e8b7c3a - default default] Volume volume-2c44a5de-50ea-4fca-87e4-7bfa1531030b (2c44a5de-50ea-4fca-87e4-7bfa1531030b): created successfully >2018-08-22 06:50:33.223 39124 INFO cinder.volume.manager [req-d98136d4-39ff-4a16-99d3-ed2fcf8fe462 4f0c9e8fc02e4045b7a37d9b34213b47 656613c3c72f40a495ee8b404e8b7c3a - default default] Created volume successfully. >2018-08-22 06:50:36.099 39124 INFO cinder.volume.manager [req-bbbb9701-5d47-4780-8380-f665c03ce753 4f0c9e8fc02e4045b7a37d9b34213b47 656613c3c72f40a495ee8b404e8b7c3a - default default] Create snapshot completed successfully >2018-08-22 06:50:38.597 39124 INFO cinder.volume.manager [req-5417aee9-281a-4787-843f-01fd1068d793 4f0c9e8fc02e4045b7a37d9b34213b47 656613c3c72f40a495ee8b404e8b7c3a - default default] Delete snapshot completed successfully. >2018-08-22 06:50:40.138 39124 INFO cinder.volume.manager [req-c91ca1c6-cc8c-4325-bc03-ff6d117ddb98 4f0c9e8fc02e4045b7a37d9b34213b47 656613c3c72f40a495ee8b404e8b7c3a - default default] Deleted volume successfully. >2018-08-22 06:51:02.317 39124 INFO cinder.volume.flows.manager.create_volume [req-ad5c2473-d71c-4400-8251-195635c57d4d ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume bfc0da4b-f2dd-48d4-8c72-292b36a99c5a: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-bfc0da4b-f2dd-48d4-8c72-292b36a99c5a', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b1d67a50>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 06:51:03.142 39124 INFO cinder.image.image_utils [req-ad5c2473-d71c-4400-8251-195635c57d4d ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 06:51:13.674 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 06:51:19.078 39124 INFO cinder.image.image_utils [req-ad5c2473-d71c-4400-8251-195635c57d4d ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Converted 39.00 MB image at 31.05 MB/s >2018-08-22 06:51:21.644 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-ad5c2473-d71c-4400-8251-195635c57d4d ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 06:51:22.226 39124 INFO cinder.volume.flows.manager.create_volume [req-ad5c2473-d71c-4400-8251-195635c57d4d ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume volume-bfc0da4b-f2dd-48d4-8c72-292b36a99c5a (bfc0da4b-f2dd-48d4-8c72-292b36a99c5a): created successfully >2018-08-22 06:51:22.231 39124 INFO cinder.volume.manager [req-ad5c2473-d71c-4400-8251-195635c57d4d ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Created volume successfully. >2018-08-22 06:51:23.527 39124 INFO cinder.volume.flows.manager.create_volume [req-0fa8ee6e-5b7d-453b-8a5a-d58b602e3562 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume 26e60a65-0b0a-49f4-bb7b-55036557e431: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-26e60a65-0b0a-49f4-bb7b-55036557e431', 'source_volid': 'bfc0da4b-f2dd-48d4-8c72-292b36a99c5a', 'source_volstatus': u'available'} >2018-08-22 06:51:25.957 39124 INFO cinder.volume.flows.manager.create_volume [req-0fa8ee6e-5b7d-453b-8a5a-d58b602e3562 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume volume-26e60a65-0b0a-49f4-bb7b-55036557e431 (26e60a65-0b0a-49f4-bb7b-55036557e431): created successfully >2018-08-22 06:51:25.961 39124 INFO cinder.volume.manager [req-0fa8ee6e-5b7d-453b-8a5a-d58b602e3562 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Created volume successfully. >2018-08-22 06:51:27.124 39124 INFO cinder.volume.flows.manager.create_volume [req-d3d76db8-dde8-4c83-9556-dacccd84acb7 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume f59ef67f-905f-46b2-9728-e65d7035a931: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f59ef67f-905f-46b2-9728-e65d7035a931'} >2018-08-22 06:51:28.389 39124 INFO cinder.volume.flows.manager.create_volume [req-d3d76db8-dde8-4c83-9556-dacccd84acb7 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume volume-f59ef67f-905f-46b2-9728-e65d7035a931 (f59ef67f-905f-46b2-9728-e65d7035a931): created successfully >2018-08-22 06:51:28.393 39124 INFO cinder.volume.manager [req-d3d76db8-dde8-4c83-9556-dacccd84acb7 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Created volume successfully. >2018-08-22 06:51:29.603 39124 INFO cinder.volume.flows.manager.create_volume [req-def712f4-4077-4222-8a92-03748f77682e ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume b5b8205e-9bf5-4430-8606-779b19f8a157: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-b5b8205e-9bf5-4430-8606-779b19f8a157', 'source_volid': 'f59ef67f-905f-46b2-9728-e65d7035a931', 'source_volstatus': u'available'} >2018-08-22 06:51:51.338 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-def712f4-4077-4222-8a92-03748f77682e ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Unable to get remote copy information for volume volume-b5b8205e-9bf5-4430-8606-779b19f8a157 >2018-08-22 06:51:51.796 39124 INFO cinder.volume.flows.manager.create_volume [req-def712f4-4077-4222-8a92-03748f77682e ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Volume volume-b5b8205e-9bf5-4430-8606-779b19f8a157 (b5b8205e-9bf5-4430-8606-779b19f8a157): created successfully >2018-08-22 06:51:51.800 39124 INFO cinder.volume.manager [req-def712f4-4077-4222-8a92-03748f77682e ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Created volume successfully. >2018-08-22 06:51:57.101 39124 INFO cinder.volume.manager [req-de35efcb-3e53-4f51-9096-3bf603a19491 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Deleted volume successfully. >2018-08-22 06:51:57.156 39124 INFO cinder.volume.manager [req-19abc0ed-768a-4f9c-86c6-888e3c406aeb ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Deleted volume successfully. >2018-08-22 06:51:57.512 39124 INFO cinder.volume.manager [req-80ab81ba-f799-4418-96a5-4057e9ecfcde ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Deleted volume successfully. >2018-08-22 06:51:57.712 39124 INFO cinder.volume.manager [req-e2f1ebfc-cf24-43fb-9e63-1c523c339b99 ec7e16246d6345879b4f52c755645860 85249c4eea514580a092a436a57fe99d - default default] Deleted volume successfully. >2018-08-22 06:52:02.011 39124 INFO cinder.volume.flows.manager.create_volume [req-425581dc-7e1d-4229-8056-699643688852 ac99186a5c934d308158a778a1dc510a dce7e58c92894902bbcd7417c1f32ca9 - default default] Volume 5a001804-0a0e-47fc-8c50-ad25bb2f5c42: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-5a001804-0a0e-47fc-8c50-ad25bb2f5c42'} >2018-08-22 06:52:02.805 39124 INFO cinder.volume.flows.manager.create_volume [req-425581dc-7e1d-4229-8056-699643688852 ac99186a5c934d308158a778a1dc510a dce7e58c92894902bbcd7417c1f32ca9 - default default] Volume volume-5a001804-0a0e-47fc-8c50-ad25bb2f5c42 (5a001804-0a0e-47fc-8c50-ad25bb2f5c42): created successfully >2018-08-22 06:52:02.809 39124 INFO cinder.volume.manager [req-425581dc-7e1d-4229-8056-699643688852 ac99186a5c934d308158a778a1dc510a dce7e58c92894902bbcd7417c1f32ca9 - default default] Created volume successfully. >2018-08-22 06:52:06.066 39124 INFO cinder.volume.manager [req-a5bb41d6-329b-48ca-9154-c58bdb13a184 ac99186a5c934d308158a778a1dc510a dce7e58c92894902bbcd7417c1f32ca9 - default default] Deleted volume successfully. >2018-08-22 06:52:29.585 39124 INFO cinder.volume.flows.manager.create_volume [req-2f230228-b0f4-45fb-b8c2-5fb5f0a829ce 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] Volume d5769dcb-6522-4a9c-8ff8-429e150e337e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d5769dcb-6522-4a9c-8ff8-429e150e337e'} >2018-08-22 06:52:30.114 39124 INFO cinder.volume.flows.manager.create_volume [req-2f230228-b0f4-45fb-b8c2-5fb5f0a829ce 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] Volume volume-d5769dcb-6522-4a9c-8ff8-429e150e337e (d5769dcb-6522-4a9c-8ff8-429e150e337e): created successfully >2018-08-22 06:52:30.118 39124 INFO cinder.volume.manager [req-2f230228-b0f4-45fb-b8c2-5fb5f0a829ce 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] Created volume successfully. >2018-08-22 06:52:32.451 39124 INFO cinder.volume.manager [req-093334ed-2522-47b6-ae92-2df52a2f0b05 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] Deleted volume successfully. >2018-08-22 06:52:32.984 39124 INFO cinder.volume.flows.manager.create_volume [req-55b54973-fdc8-47c6-9593-2b0a522e40aa 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] Volume 493281f7-2b04-4962-a89e-770a46bf12d1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-493281f7-2b04-4962-a89e-770a46bf12d1'} >2018-08-22 06:52:33.512 39124 INFO cinder.volume.flows.manager.create_volume [req-55b54973-fdc8-47c6-9593-2b0a522e40aa 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] Volume volume-493281f7-2b04-4962-a89e-770a46bf12d1 (493281f7-2b04-4962-a89e-770a46bf12d1): created successfully >2018-08-22 06:52:33.516 39124 INFO cinder.volume.manager [req-55b54973-fdc8-47c6-9593-2b0a522e40aa 72847e565bd745ed9f8179ecdd9f1073 19a5f7aeaa394f8b9db63770d14ca9f4 - default default] Created volume successfully. >2018-08-22 06:52:35.322 39124 INFO cinder.volume.manager [req-de505afd-52fb-401a-a891-b013eed72490 d17f1133c5f04bd4bd3a0760960ca604 125b281fa13c404db6b7b5c5962c11d4 - default default] Transfer volume completed successfully. >2018-08-22 06:52:37.287 39124 INFO cinder.volume.manager [req-7f927445-7c01-414e-83a2-f64d52c0298f 43bec6ddf9c34d5c9576d7d9ded29423 589e0677869449dbaec11db5601f8589 - default default] Deleted volume successfully. >2018-08-22 07:01:28.680 39124 INFO cinder.volume.manager [req-3c0dd6a8-ba08-426a-a15a-0161adf60064 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-485853078: creating >2018-08-22 07:01:28.708 39124 INFO cinder.volume.manager [req-3c0dd6a8-ba08-426a-a15a-0161adf60064 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-485853078: created successfully >2018-08-22 07:01:28.730 39124 INFO cinder.volume.manager [req-3c0dd6a8-ba08-426a-a15a-0161adf60064 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Create group completed successfully. >2018-08-22 07:01:29.832 39124 INFO cinder.volume.flows.manager.create_volume [req-2fe99989-129e-4f3e-a1a6-04b7a90ee780 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume e7afaeba-6811-4ed0-9f8f-18d98226610a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e7afaeba-6811-4ed0-9f8f-18d98226610a'} >2018-08-22 07:01:30.377 39124 INFO cinder.volume.flows.manager.create_volume [req-2fe99989-129e-4f3e-a1a6-04b7a90ee780 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume volume-e7afaeba-6811-4ed0-9f8f-18d98226610a (e7afaeba-6811-4ed0-9f8f-18d98226610a): created successfully >2018-08-22 07:01:30.381 39124 INFO cinder.volume.manager [req-2fe99989-129e-4f3e-a1a6-04b7a90ee780 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Created volume successfully. >2018-08-22 07:01:31.743 39124 INFO cinder.volume.manager [req-29527534-0198-4324-a534-63974c6cfb60 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] GroupSnapshot c4601d7d-b9c9-456f-b15a-c4a88801382b: creating. >2018-08-22 07:01:33.827 39124 INFO cinder.volume.manager [req-29527534-0198-4324-a534-63974c6cfb60 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] group_snapshot c4601d7d-b9c9-456f-b15a-c4a88801382b: created successfully >2018-08-22 07:01:34.692 39124 INFO cinder.volume.manager [req-ae51311d-67d6-4b52-b693-52dff8aba639 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] group_snapshot c4601d7d-b9c9-456f-b15a-c4a88801382b: deleting >2018-08-22 07:01:36.487 39124 WARNING cinder.quota [req-ae51311d-67d6-4b52-b693-52dff8aba639 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-1044327785 is set by the default quota flag: quota_snapshots_tempest-volume-type-1044327785, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:36.517 39124 INFO cinder.volume.manager [req-ae51311d-67d6-4b52-b693-52dff8aba639 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] group_snapshot c4601d7d-b9c9-456f-b15a-c4a88801382b: deleted successfully >2018-08-22 07:01:38.780 39124 WARNING cinder.quota [req-a253882a-438b-437f-a8d1-11613bd975a9 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1044327785 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1044327785, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:38.781 39124 WARNING cinder.quota [req-a253882a-438b-437f-a8d1-11613bd975a9 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1044327785 is set by the default quota flag: quota_volumes_tempest-volume-type-1044327785, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:39.052 39124 INFO cinder.volume.manager [req-a253882a-438b-437f-a8d1-11613bd975a9 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Delete group completed successfully. >2018-08-22 07:01:39.478 39124 INFO cinder.volume.manager [req-78faa4fa-57dc-4b05-b11c-a89f2292723c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-1341579112: creating >2018-08-22 07:01:39.500 39124 INFO cinder.volume.manager [req-78faa4fa-57dc-4b05-b11c-a89f2292723c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-1341579112: created successfully >2018-08-22 07:01:39.520 39124 INFO cinder.volume.manager [req-78faa4fa-57dc-4b05-b11c-a89f2292723c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Create group completed successfully. >2018-08-22 07:01:39.884 39124 INFO cinder.volume.flows.manager.create_volume [req-edb8b4b6-2313-4818-8339-c5fb0f7b28ae 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume cfe54046-726f-426c-9810-400e62f482ef: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cfe54046-726f-426c-9810-400e62f482ef'} >2018-08-22 07:01:40.430 39124 INFO cinder.volume.flows.manager.create_volume [req-edb8b4b6-2313-4818-8339-c5fb0f7b28ae 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume volume-cfe54046-726f-426c-9810-400e62f482ef (cfe54046-726f-426c-9810-400e62f482ef): created successfully >2018-08-22 07:01:40.434 39124 INFO cinder.volume.manager [req-edb8b4b6-2313-4818-8339-c5fb0f7b28ae 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Created volume successfully. >2018-08-22 07:01:42.073 39124 WARNING cinder.quota [req-a6d5a54a-afb6-444c-875a-6f48bae82a77 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1691726502 is set by the default quota flag: quota_volumes_tempest-volume-type-1691726502, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:42.075 39124 WARNING cinder.quota [req-a6d5a54a-afb6-444c-875a-6f48bae82a77 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1691726502 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1691726502, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:42.356 39124 INFO cinder.volume.manager [req-a6d5a54a-afb6-444c-875a-6f48bae82a77 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Delete group completed successfully. >2018-08-22 07:01:42.621 39124 INFO cinder.volume.manager [req-0ba9e497-008e-4e44-b9ef-67011144ad7e 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-500241463: creating >2018-08-22 07:01:42.642 39124 INFO cinder.volume.manager [req-0ba9e497-008e-4e44-b9ef-67011144ad7e 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-500241463: created successfully >2018-08-22 07:01:42.663 39124 INFO cinder.volume.manager [req-0ba9e497-008e-4e44-b9ef-67011144ad7e 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Create group completed successfully. >2018-08-22 07:01:43.046 39124 INFO cinder.volume.flows.manager.create_volume [req-8b65007d-8062-4fae-abaa-8490338472c5 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume e7197881-0888-4a47-9b22-b296b0c4271e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e7197881-0888-4a47-9b22-b296b0c4271e'} >2018-08-22 07:01:43.590 39124 INFO cinder.volume.flows.manager.create_volume [req-8b65007d-8062-4fae-abaa-8490338472c5 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume volume-e7197881-0888-4a47-9b22-b296b0c4271e (e7197881-0888-4a47-9b22-b296b0c4271e): created successfully >2018-08-22 07:01:43.593 39124 INFO cinder.volume.manager [req-8b65007d-8062-4fae-abaa-8490338472c5 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Created volume successfully. >2018-08-22 07:01:44.272 39124 INFO cinder.volume.manager [req-9e279525-1942-44a7-9f00-42a37b5655ad 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] GroupSnapshot a6d2cca0-0732-4097-8af1-4304e8df6fef: creating. >2018-08-22 07:01:46.573 39124 INFO cinder.volume.manager [req-9e279525-1942-44a7-9f00-42a37b5655ad 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] group_snapshot a6d2cca0-0732-4097-8af1-4304e8df6fef: created successfully >2018-08-22 07:01:49.404 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-ab161bca-19b6-4045-a824-034b7b79ebe7 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Updating status for CG: a8ed711e-ebfe-40b3-92c3-6761c73c6ddc. >2018-08-22 07:01:49.516 39124 INFO cinder.volume.manager [req-ab161bca-19b6-4045-a824-034b7b79ebe7 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Create group from source-snapshot-a6d2cca0-0732-4097-8af1-4304e8df6fef completed successfully. >2018-08-22 07:01:54.930 39124 WARNING cinder.quota [req-63730bb6-9f9e-4c0a-ab3c-08008a414587 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1434592450 is set by the default quota flag: quota_volumes_tempest-volume-type-1434592450, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:54.933 39124 WARNING cinder.quota [req-63730bb6-9f9e-4c0a-ab3c-08008a414587 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1434592450 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1434592450, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:55.205 39124 INFO cinder.volume.manager [req-63730bb6-9f9e-4c0a-ab3c-08008a414587 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Delete group completed successfully. >2018-08-22 07:01:55.624 39124 INFO cinder.volume.manager [req-9aab3829-3309-4f80-a7a3-6a4bf455b6b9 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] group_snapshot a6d2cca0-0732-4097-8af1-4304e8df6fef: deleting >2018-08-22 07:01:57.808 39124 WARNING cinder.quota [req-9aab3829-3309-4f80-a7a3-6a4bf455b6b9 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-1434592450 is set by the default quota flag: quota_snapshots_tempest-volume-type-1434592450, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:01:57.841 39124 INFO cinder.volume.manager [req-9aab3829-3309-4f80-a7a3-6a4bf455b6b9 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] group_snapshot a6d2cca0-0732-4097-8af1-4304e8df6fef: deleted successfully >2018-08-22 07:01:59.425 39124 INFO cinder.volume.manager [req-73c0efc8-feac-4eaf-9a5e-1c6128ee5463 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Delete group completed successfully. >2018-08-22 07:01:59.749 39124 INFO cinder.volume.manager [req-d31d450d-fd04-4f66-8d2f-909698c0449c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-475104457: creating >2018-08-22 07:01:59.776 39124 INFO cinder.volume.manager [req-d31d450d-fd04-4f66-8d2f-909698c0449c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Group tempest-CG-475104457: created successfully >2018-08-22 07:01:59.795 39124 INFO cinder.volume.manager [req-d31d450d-fd04-4f66-8d2f-909698c0449c 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Create group completed successfully. >2018-08-22 07:02:00.147 39124 INFO cinder.volume.flows.manager.create_volume [req-014899ef-d5f3-49ba-93c3-ee13de28f67f 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume f456ea33-d40c-4f3d-9be7-f17866a613b8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f456ea33-d40c-4f3d-9be7-f17866a613b8'} >2018-08-22 07:02:00.690 39124 INFO cinder.volume.flows.manager.create_volume [req-014899ef-d5f3-49ba-93c3-ee13de28f67f 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Volume volume-f456ea33-d40c-4f3d-9be7-f17866a613b8 (f456ea33-d40c-4f3d-9be7-f17866a613b8): created successfully >2018-08-22 07:02:00.693 39124 INFO cinder.volume.manager [req-014899ef-d5f3-49ba-93c3-ee13de28f67f 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Created volume successfully. >2018-08-22 07:02:03.845 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c7b5ec9f-e741-402d-8fac-1e6cb24dffec 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Updating status for CG: 05668c44-378a-4f4a-af27-5503ef282ee3. >2018-08-22 07:02:03.946 39124 INFO cinder.volume.manager [req-c7b5ec9f-e741-402d-8fac-1e6cb24dffec 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Create group from source-group-73a97b73-8665-42f4-8f57-32f5d2da3ff5 completed successfully. >2018-08-22 07:02:08.157 39124 WARNING cinder.quota [req-89d86a00-da52-4d5d-a905-9b08d620efce 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1350395280 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1350395280, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:02:08.159 39124 WARNING cinder.quota [req-89d86a00-da52-4d5d-a905-9b08d620efce 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1350395280 is set by the default quota flag: quota_volumes_tempest-volume-type-1350395280, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:02:08.447 39124 INFO cinder.volume.manager [req-89d86a00-da52-4d5d-a905-9b08d620efce 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Delete group completed successfully. >2018-08-22 07:02:10.391 39124 INFO cinder.volume.manager [req-8b8062a6-4dbc-4826-9c43-49466e2d4516 4fce86cb0eb04ef4b559fdbaffcd1f23 db90958ef7574d35a0d3eb87a0c74fb7 - default default] Delete group completed successfully. >2018-08-22 07:03:21.130 39124 INFO cinder.volume.flows.manager.create_volume [req-ffd5a2a8-fd5c-4c83-9fad-353c68a7449e 69680fa5661a46b3abf1997652f42db7 9ca78b287d454e60a66d5f17600faa4f - default default] Volume c67eeff4-aa2f-4243-b6fa-f325cac866e9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c67eeff4-aa2f-4243-b6fa-f325cac866e9'} >2018-08-22 07:03:21.950 39124 INFO cinder.volume.flows.manager.create_volume [req-ffd5a2a8-fd5c-4c83-9fad-353c68a7449e 69680fa5661a46b3abf1997652f42db7 9ca78b287d454e60a66d5f17600faa4f - default default] Volume volume-c67eeff4-aa2f-4243-b6fa-f325cac866e9 (c67eeff4-aa2f-4243-b6fa-f325cac866e9): created successfully >2018-08-22 07:03:21.954 39124 INFO cinder.volume.manager [req-ffd5a2a8-fd5c-4c83-9fad-353c68a7449e 69680fa5661a46b3abf1997652f42db7 9ca78b287d454e60a66d5f17600faa4f - default default] Created volume successfully. >2018-08-22 07:03:24.601 39124 INFO cinder.volume.manager [req-ae154149-f704-41c0-86c2-bbf1a8e55538 69680fa5661a46b3abf1997652f42db7 9ca78b287d454e60a66d5f17600faa4f - default default] Deleted volume successfully. >2018-08-22 07:03:33.466 39124 INFO cinder.volume.flows.manager.create_volume [req-ddce8255-83fa-4ffc-8d18-d0c820ba3466 704a74104faf4d2bad90a7b0055a1863 6a6894129f244ecbb71dd3fa496974bb - default default] Volume b249a6fb-0b0d-4022-a87b-334e62492f1d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b249a6fb-0b0d-4022-a87b-334e62492f1d'} >2018-08-22 07:03:34.007 39124 INFO cinder.volume.flows.manager.create_volume [req-ddce8255-83fa-4ffc-8d18-d0c820ba3466 704a74104faf4d2bad90a7b0055a1863 6a6894129f244ecbb71dd3fa496974bb - default default] Volume volume-b249a6fb-0b0d-4022-a87b-334e62492f1d (b249a6fb-0b0d-4022-a87b-334e62492f1d): created successfully >2018-08-22 07:03:34.011 39124 INFO cinder.volume.manager [req-ddce8255-83fa-4ffc-8d18-d0c820ba3466 704a74104faf4d2bad90a7b0055a1863 6a6894129f244ecbb71dd3fa496974bb - default default] Created volume successfully. >2018-08-22 07:03:37.502 39124 WARNING cinder.quota [req-2982762d-04e0-4a20-bba6-1c36150810d2 704a74104faf4d2bad90a7b0055a1863 6a6894129f244ecbb71dd3fa496974bb - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-410519570 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-410519570, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:03:37.503 39124 WARNING cinder.quota [req-2982762d-04e0-4a20-bba6-1c36150810d2 704a74104faf4d2bad90a7b0055a1863 6a6894129f244ecbb71dd3fa496974bb - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-410519570 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-410519570, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:03:37.753 39124 INFO cinder.volume.manager [req-2982762d-04e0-4a20-bba6-1c36150810d2 704a74104faf4d2bad90a7b0055a1863 6a6894129f244ecbb71dd3fa496974bb - default default] Deleted volume successfully. >2018-08-22 07:03:46.771 39124 INFO cinder.volume.flows.manager.create_volume [req-0fcd7d69-363f-4d5d-a120-9284cf241fff a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] Volume 75d53054-2d57-4eab-9569-efda1f160f87: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-75d53054-2d57-4eab-9569-efda1f160f87'} >2018-08-22 07:03:47.412 39124 INFO cinder.volume.flows.manager.create_volume [req-0fcd7d69-363f-4d5d-a120-9284cf241fff a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] Volume volume-75d53054-2d57-4eab-9569-efda1f160f87 (75d53054-2d57-4eab-9569-efda1f160f87): created successfully >2018-08-22 07:03:47.416 39124 INFO cinder.volume.manager [req-0fcd7d69-363f-4d5d-a120-9284cf241fff a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] Created volume successfully. >2018-08-22 07:03:48.912 39124 INFO cinder.volume.manager [req-e2975fa7-251f-4159-8af4-6c911a71f1a7 a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] Retype volume completed successfully. >2018-08-22 07:03:52.726 39124 WARNING cinder.quota [req-26ab3bb7-8517-445e-ba81-ff103c0e614d a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1142648325 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1142648325, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:03:52.728 39124 WARNING cinder.quota [req-26ab3bb7-8517-445e-ba81-ff103c0e614d a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1142648325 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1142648325, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:03:52.989 39124 INFO cinder.volume.manager [req-26ab3bb7-8517-445e-ba81-ff103c0e614d a79faf59ec7b491d901d18716e75ef53 0152cd04166e4eeca6f36a3647ccdc23 - default default] Deleted volume successfully. >2018-08-22 07:04:33.000 39124 INFO cinder.volume.flows.manager.create_volume [req-c22b7efd-d9cd-436b-ab13-0f26fad587a5 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume f39a915a-5983-4332-83cb-29ef8e580dbd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f39a915a-5983-4332-83cb-29ef8e580dbd'} >2018-08-22 07:04:33.532 39124 INFO cinder.volume.flows.manager.create_volume [req-c22b7efd-d9cd-436b-ab13-0f26fad587a5 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume volume-f39a915a-5983-4332-83cb-29ef8e580dbd (f39a915a-5983-4332-83cb-29ef8e580dbd): created successfully >2018-08-22 07:04:33.536 39124 INFO cinder.volume.manager [req-c22b7efd-d9cd-436b-ab13-0f26fad587a5 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Created volume successfully. >2018-08-22 07:04:34.725 39124 INFO cinder.volume.manager [req-dd089ecb-2401-4459-a1ee-d05710353543 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Attaching volume f39a915a-5983-4332-83cb-29ef8e580dbd to instance 0addf349-a925-446d-957f-32c6b3a5a4e7 at mountpoint /dev/vdb on host None. >2018-08-22 07:04:34.798 39124 INFO cinder.volume.manager [req-dd089ecb-2401-4459-a1ee-d05710353543 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Attach volume completed successfully. >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager [req-10e3beb3-119c-45ce-a9a1-3cc1799bfa03 b7edbb50693b4930aef26055ff95b2a9 6ea81a24495943d1a101df2d5644ae2c - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager force=force) >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-22 07:04:36.156 39124 ERROR cinder.volume.manager >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server [req-10e3beb3-119c-45ce-a9a1-3cc1799bfa03 b7edbb50693b4930aef26055ff95b2a9 6ea81a24495943d1a101df2d5644ae2c - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-22 07:04:36.158 39124 ERROR oslo_messaging.rpc.server >2018-08-22 07:04:36.278 39124 INFO cinder.volume.manager [req-3a6474e0-ed1b-4524-b276-50d2f988f435 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Detaching volume f39a915a-5983-4332-83cb-29ef8e580dbd from instance 0addf349-a925-446d-957f-32c6b3a5a4e7. >2018-08-22 07:04:36.364 39124 INFO cinder.volume.manager [req-3a6474e0-ed1b-4524-b276-50d2f988f435 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Detach volume completed successfully. >2018-08-22 07:04:38.314 39116 INFO cinder.volume.flows.manager.create_volume [req-64d6fa32-b9e1-415f-b603-2a2fa50d6153 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume 6be004dd-d1ab-4fc2-937e-f9c3f97853d6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6be004dd-d1ab-4fc2-937e-f9c3f97853d6'} >2018-08-22 07:04:38.848 39116 INFO cinder.volume.flows.manager.create_volume [req-64d6fa32-b9e1-415f-b603-2a2fa50d6153 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume volume-6be004dd-d1ab-4fc2-937e-f9c3f97853d6 (6be004dd-d1ab-4fc2-937e-f9c3f97853d6): created successfully >2018-08-22 07:04:38.852 39116 INFO cinder.volume.manager [req-64d6fa32-b9e1-415f-b603-2a2fa50d6153 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Created volume successfully. >2018-08-22 07:04:41.035 39116 INFO cinder.volume.manager [req-83ab4bfa-d5cb-481f-af0c-9d0ee441bcdc b7edbb50693b4930aef26055ff95b2a9 6ea81a24495943d1a101df2d5644ae2c - default default] Deleted volume successfully. >2018-08-22 07:04:41.924 39124 INFO cinder.volume.flows.manager.create_volume [req-d2833f81-a212-4bf7-957f-dc7b482e4d7a b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume 3918088d-5baf-4cca-9a86-add4aa082430: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3918088d-5baf-4cca-9a86-add4aa082430'} >2018-08-22 07:04:42.451 39124 INFO cinder.volume.flows.manager.create_volume [req-d2833f81-a212-4bf7-957f-dc7b482e4d7a b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume volume-3918088d-5baf-4cca-9a86-add4aa082430 (3918088d-5baf-4cca-9a86-add4aa082430): created successfully >2018-08-22 07:04:42.455 39124 INFO cinder.volume.manager [req-d2833f81-a212-4bf7-957f-dc7b482e4d7a b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Created volume successfully. >2018-08-22 07:04:44.871 39124 INFO cinder.volume.manager [req-bb6d1a01-fbe1-4a3e-963d-745e77a9a48f b7edbb50693b4930aef26055ff95b2a9 6ea81a24495943d1a101df2d5644ae2c - default default] Deleted volume successfully. >2018-08-22 07:04:45.254 39124 INFO cinder.volume.flows.manager.create_volume [req-bb38ac1a-4fb8-4905-a62e-aa10b251ac1b b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume fd249904-7c62-4650-86b4-89d4b95393de: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fd249904-7c62-4650-86b4-89d4b95393de'} >2018-08-22 07:04:45.797 39124 INFO cinder.volume.flows.manager.create_volume [req-bb38ac1a-4fb8-4905-a62e-aa10b251ac1b b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume volume-fd249904-7c62-4650-86b4-89d4b95393de (fd249904-7c62-4650-86b4-89d4b95393de): created successfully >2018-08-22 07:04:45.801 39124 INFO cinder.volume.manager [req-bb38ac1a-4fb8-4905-a62e-aa10b251ac1b b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Created volume successfully. >2018-08-22 07:04:48.332 39124 INFO cinder.volume.manager [req-8281cb42-e75c-4e88-b2ec-7bb93a87d98c b7edbb50693b4930aef26055ff95b2a9 6ea81a24495943d1a101df2d5644ae2c - default default] Deleted volume successfully. >2018-08-22 07:04:48.573 39124 INFO cinder.volume.flows.manager.create_volume [req-f6cda490-f21a-47d4-be93-79ef99f7520c b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume 1e43db0e-ce94-4548-892a-3c5e99bdcd5b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1e43db0e-ce94-4548-892a-3c5e99bdcd5b'} >2018-08-22 07:04:49.105 39124 INFO cinder.volume.flows.manager.create_volume [req-f6cda490-f21a-47d4-be93-79ef99f7520c b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume volume-1e43db0e-ce94-4548-892a-3c5e99bdcd5b (1e43db0e-ce94-4548-892a-3c5e99bdcd5b): created successfully >2018-08-22 07:04:49.109 39124 INFO cinder.volume.manager [req-f6cda490-f21a-47d4-be93-79ef99f7520c b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Created volume successfully. >2018-08-22 07:04:51.107 39124 INFO cinder.volume.manager [req-f709f985-b4cf-4c75-9c80-a292eb1fbd62 b7edbb50693b4930aef26055ff95b2a9 6ea81a24495943d1a101df2d5644ae2c - default default] Deleted volume successfully. >2018-08-22 07:04:51.569 39124 INFO cinder.volume.flows.manager.create_volume [req-79194661-60d5-43d9-a05a-434992f2e9fd b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume 92c9909d-7346-4c46-9206-f5b6d6716c90: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-92c9909d-7346-4c46-9206-f5b6d6716c90'} >2018-08-22 07:04:52.107 39124 INFO cinder.volume.flows.manager.create_volume [req-79194661-60d5-43d9-a05a-434992f2e9fd b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Volume volume-92c9909d-7346-4c46-9206-f5b6d6716c90 (92c9909d-7346-4c46-9206-f5b6d6716c90): created successfully >2018-08-22 07:04:52.112 39124 INFO cinder.volume.manager [req-79194661-60d5-43d9-a05a-434992f2e9fd b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Created volume successfully. >2018-08-22 07:04:54.870 39124 INFO cinder.volume.manager [req-6518fe70-66d9-4214-a6c5-929ff39955b9 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Deleted volume successfully. >2018-08-22 07:04:55.640 39124 INFO cinder.volume.manager [req-6c412042-9766-4c3c-a17f-9f1579c7e9f8 b9541e0b420a4dfe9c9d170d3483801a 358ea4c2a21643be898e2aed93579074 - default default] Deleted volume successfully. >2018-08-22 07:05:03.615 39124 INFO cinder.volume.flows.manager.create_volume [req-15ee3154-11e3-40b8-8e85-0bc4dfc7fb42 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Volume 5df703b8-ff04-4355-8fe9-9d3e6707a676: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5df703b8-ff04-4355-8fe9-9d3e6707a676'} >2018-08-22 07:05:04.163 39124 INFO cinder.volume.flows.manager.create_volume [req-15ee3154-11e3-40b8-8e85-0bc4dfc7fb42 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Volume volume-5df703b8-ff04-4355-8fe9-9d3e6707a676 (5df703b8-ff04-4355-8fe9-9d3e6707a676): created successfully >2018-08-22 07:05:04.167 39124 INFO cinder.volume.manager [req-15ee3154-11e3-40b8-8e85-0bc4dfc7fb42 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Created volume successfully. >2018-08-22 07:05:05.658 39124 INFO cinder.volume.flows.manager.create_volume [req-0c8ae1dc-3878-49e2-8956-e33b1241031f 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Volume 0c6d7d37-adcc-4d0b-988f-5f4ef2a2eed6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0c6d7d37-adcc-4d0b-988f-5f4ef2a2eed6'} >2018-08-22 07:05:06.251 39124 INFO cinder.volume.flows.manager.create_volume [req-0c8ae1dc-3878-49e2-8956-e33b1241031f 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Volume volume-0c6d7d37-adcc-4d0b-988f-5f4ef2a2eed6 (0c6d7d37-adcc-4d0b-988f-5f4ef2a2eed6): created successfully >2018-08-22 07:05:06.255 39124 INFO cinder.volume.manager [req-0c8ae1dc-3878-49e2-8956-e33b1241031f 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Created volume successfully. >2018-08-22 07:05:07.685 39116 INFO cinder.volume.flows.manager.create_volume [req-dbcb84b1-9977-4dcb-a9e3-a2c97116528d 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Volume c710e895-374f-4c74-a0c0-91ac154ef7e8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c710e895-374f-4c74-a0c0-91ac154ef7e8'} >2018-08-22 07:05:08.327 39116 INFO cinder.volume.flows.manager.create_volume [req-dbcb84b1-9977-4dcb-a9e3-a2c97116528d 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Volume volume-c710e895-374f-4c74-a0c0-91ac154ef7e8 (c710e895-374f-4c74-a0c0-91ac154ef7e8): created successfully >2018-08-22 07:05:08.330 39116 INFO cinder.volume.manager [req-dbcb84b1-9977-4dcb-a9e3-a2c97116528d 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Created volume successfully. >2018-08-22 07:05:09.517 39124 INFO cinder.volume.flows.manager.create_volume [req-cf94f4dc-7432-4cf3-bf60-31ab887378e9 e567d95c459d4e30b47714e3bb044681 a8fb05dd7cd943da86ed411eff53e973 - default default] Volume e5d629f9-98d0-4133-bac1-1dbcf7e92a2e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e5d629f9-98d0-4133-bac1-1dbcf7e92a2e'} >2018-08-22 07:05:10.066 39124 INFO cinder.volume.flows.manager.create_volume [req-cf94f4dc-7432-4cf3-bf60-31ab887378e9 e567d95c459d4e30b47714e3bb044681 a8fb05dd7cd943da86ed411eff53e973 - default default] Volume volume-e5d629f9-98d0-4133-bac1-1dbcf7e92a2e (e5d629f9-98d0-4133-bac1-1dbcf7e92a2e): created successfully >2018-08-22 07:05:10.070 39124 INFO cinder.volume.manager [req-cf94f4dc-7432-4cf3-bf60-31ab887378e9 e567d95c459d4e30b47714e3bb044681 a8fb05dd7cd943da86ed411eff53e973 - default default] Created volume successfully. >2018-08-22 07:05:14.798 39124 INFO cinder.volume.manager [req-41288dc6-c725-49c3-9194-4c0e1d808e8c e567d95c459d4e30b47714e3bb044681 a8fb05dd7cd943da86ed411eff53e973 - default default] Deleted volume successfully. >2018-08-22 07:05:14.817 39124 INFO cinder.volume.manager [req-04503b7f-27d2-495d-85fb-086721d8e295 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Deleted volume successfully. >2018-08-22 07:05:14.901 39124 INFO cinder.volume.manager [req-1836bbcd-146c-4d09-8b72-a7e95184f605 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Deleted volume successfully. >2018-08-22 07:05:15.105 39116 INFO cinder.volume.manager [req-68dcafcb-741a-4fc8-a346-bd6397280041 5367bf1285dd4d25814b235ca50a9540 2227c0dc4553424cbf4d4254f9987cf3 - default default] Deleted volume successfully. >2018-08-22 07:05:21.131 39124 INFO cinder.volume.flows.manager.create_volume [req-b1092a88-19c2-41ac-8e2d-aac5b20834c4 956c84cfcfec46a4b219d9dc57ed5476 caac17bf082449deaffc37b93109410e - default default] Volume ca5646ab-19c1-4d4d-85c3-306c8be70b0e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ca5646ab-19c1-4d4d-85c3-306c8be70b0e'} >2018-08-22 07:05:21.657 39124 INFO cinder.volume.flows.manager.create_volume [req-b1092a88-19c2-41ac-8e2d-aac5b20834c4 956c84cfcfec46a4b219d9dc57ed5476 caac17bf082449deaffc37b93109410e - default default] Volume volume-ca5646ab-19c1-4d4d-85c3-306c8be70b0e (ca5646ab-19c1-4d4d-85c3-306c8be70b0e): created successfully >2018-08-22 07:05:21.661 39124 INFO cinder.volume.manager [req-b1092a88-19c2-41ac-8e2d-aac5b20834c4 956c84cfcfec46a4b219d9dc57ed5476 caac17bf082449deaffc37b93109410e - default default] Created volume successfully. >2018-08-22 07:05:25.849 39124 INFO cinder.volume.manager [req-7b43690d-11c1-48de-8058-f64a24d62663 956c84cfcfec46a4b219d9dc57ed5476 caac17bf082449deaffc37b93109410e - default default] Deleted volume successfully. >2018-08-22 07:05:32.762 39124 INFO cinder.volume.flows.manager.create_volume [req-8e59745d-cc56-4870-9f28-b1d5949863d0 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] Volume 1647fe32-6ae6-42db-a420-3a0dca149655: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1647fe32-6ae6-42db-a420-3a0dca149655'} >2018-08-22 07:05:33.296 39124 INFO cinder.volume.flows.manager.create_volume [req-8e59745d-cc56-4870-9f28-b1d5949863d0 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] Volume volume-1647fe32-6ae6-42db-a420-3a0dca149655 (1647fe32-6ae6-42db-a420-3a0dca149655): created successfully >2018-08-22 07:05:33.303 39124 INFO cinder.volume.manager [req-8e59745d-cc56-4870-9f28-b1d5949863d0 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] Created volume successfully. >2018-08-22 07:05:35.468 39124 INFO cinder.volume.manager [req-140093c3-3850-43ac-ae46-8b82d6f3ff8a 5b4c5ff8bd0141eeb5a73d707f5c6119 ab186bc5130d453ea6ddaea813cd93cf - default default] Transfer volume completed successfully. >2018-08-22 07:05:37.575 39124 INFO cinder.volume.manager [req-52991983-4154-49f0-a517-e8595d4a3465 36156ce90871496db60a69a3c558b32d fad567bb8c6f4ca797a650a240d736cf - default default] Deleted volume successfully. >2018-08-22 07:05:38.746 39124 INFO cinder.volume.flows.manager.create_volume [req-8158c23c-52be-447b-a169-ac1bbd03f370 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] Volume 7bb5ffc6-7640-437c-bbfa-6ea73d9d61e4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7bb5ffc6-7640-437c-bbfa-6ea73d9d61e4'} >2018-08-22 07:05:39.557 39124 INFO cinder.volume.flows.manager.create_volume [req-8158c23c-52be-447b-a169-ac1bbd03f370 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] Volume volume-7bb5ffc6-7640-437c-bbfa-6ea73d9d61e4 (7bb5ffc6-7640-437c-bbfa-6ea73d9d61e4): created successfully >2018-08-22 07:05:39.561 39124 INFO cinder.volume.manager [req-8158c23c-52be-447b-a169-ac1bbd03f370 df97a5253b5041f38e1eec8819606a52 18369e43668044368a3ef00448a775b2 - default default] Created volume successfully. >2018-08-22 07:05:42.418 39124 INFO cinder.volume.manager [req-9d95423a-a13f-473b-8424-f79c76596c10 36156ce90871496db60a69a3c558b32d fad567bb8c6f4ca797a650a240d736cf - default default] Deleted volume successfully. >2018-08-22 07:05:49.139 39124 INFO cinder.volume.flows.manager.create_volume [req-c7d96d18-218a-49b8-9915-e9986a2d01c4 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Volume eb5017ac-4a51-4f08-902a-536b2521f036: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-eb5017ac-4a51-4f08-902a-536b2521f036'} >2018-08-22 07:05:49.950 39124 INFO cinder.volume.flows.manager.create_volume [req-c7d96d18-218a-49b8-9915-e9986a2d01c4 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Volume volume-eb5017ac-4a51-4f08-902a-536b2521f036 (eb5017ac-4a51-4f08-902a-536b2521f036): created successfully >2018-08-22 07:05:49.954 39124 INFO cinder.volume.manager [req-c7d96d18-218a-49b8-9915-e9986a2d01c4 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Created volume successfully. >2018-08-22 07:05:55.880 39124 INFO cinder.volume.manager [req-3f97e65c-5954-46a7-b065-9f41a26ad898 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Attaching volume eb5017ac-4a51-4f08-902a-536b2521f036 to instance 7d75d4fb-2319-4124-93e5-4b0ee59becd5 at mountpoint /dev/vdb on host None. >2018-08-22 07:05:55.956 39124 INFO cinder.volume.manager [req-3f97e65c-5954-46a7-b065-9f41a26ad898 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Attach volume completed successfully. >2018-08-22 07:05:56.624 39124 INFO cinder.volume.manager [req-243f7f1b-f77c-4e2c-86b4-180382df7a45 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Detaching volume eb5017ac-4a51-4f08-902a-536b2521f036 from instance 7d75d4fb-2319-4124-93e5-4b0ee59becd5. >2018-08-22 07:05:56.707 39124 INFO cinder.volume.manager [req-243f7f1b-f77c-4e2c-86b4-180382df7a45 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Detach volume completed successfully. >2018-08-22 07:06:02.980 39124 INFO cinder.volume.manager [req-e0eac6cc-8013-48bc-a157-f2f7c7393a50 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Attaching volume eb5017ac-4a51-4f08-902a-536b2521f036 to instance 4f96348c-009b-4fb5-9412-f613127431c6 at mountpoint /dev/vdb on host None. >2018-08-22 07:06:03.052 39124 INFO cinder.volume.manager [req-e0eac6cc-8013-48bc-a157-f2f7c7393a50 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Attach volume completed successfully. >2018-08-22 07:06:03.564 39124 INFO cinder.volume.manager [req-c591d2c5-55b8-4ea0-a3b6-a4d562b9fca2 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Detaching volume eb5017ac-4a51-4f08-902a-536b2521f036 from instance 4f96348c-009b-4fb5-9412-f613127431c6. >2018-08-22 07:06:03.649 39124 INFO cinder.volume.manager [req-c591d2c5-55b8-4ea0-a3b6-a4d562b9fca2 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Detach volume completed successfully. >2018-08-22 07:06:16.609 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 07:06:34.207 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d02a3ed3-a82c-4564-bf43-b93cf4463301 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:06:34.663 39124 INFO cinder.volume.manager [req-d02a3ed3-a82c-4564-bf43-b93cf4463301 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Copy volume to image completed successfully. >2018-08-22 07:06:37.404 39124 INFO cinder.volume.manager [req-d5bb5875-625c-4b1d-839a-58e287356617 0cea18dbf8b344acbeb7322990fb1342 30c79427c3cd44958b4cc0b2cdb5157f - default default] Deleted volume successfully. >2018-08-22 07:06:43.723 39124 INFO cinder.volume.flows.manager.create_volume [req-a1454209-59bd-45b8-a42f-6d85c543c0c2 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Volume 1e97788b-88c6-4f54-bc61-925fab5e608d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1e97788b-88c6-4f54-bc61-925fab5e608d'} >2018-08-22 07:06:44.675 39124 INFO cinder.volume.flows.manager.create_volume [req-a1454209-59bd-45b8-a42f-6d85c543c0c2 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Volume volume-1e97788b-88c6-4f54-bc61-925fab5e608d (1e97788b-88c6-4f54-bc61-925fab5e608d): created successfully >2018-08-22 07:06:44.679 39124 INFO cinder.volume.manager [req-a1454209-59bd-45b8-a42f-6d85c543c0c2 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Created volume successfully. >2018-08-22 07:07:08.037 39124 INFO cinder.volume.manager [req-ea8fbeea-e33b-4d3c-bee2-373f8de3f206 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Initialize volume connection completed successfully. >2018-08-22 07:07:12.741 39124 INFO cinder.volume.manager [req-886d7e4d-9d6c-4b4e-850d-0a2a8784517d 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Attaching volume 1e97788b-88c6-4f54-bc61-925fab5e608d to instance 5d59aab1-9561-43a4-b51d-15bca6b36a3f at mountpoint /dev/vdb on host None. >2018-08-22 07:07:12.932 39124 INFO cinder.volume.manager [req-886d7e4d-9d6c-4b4e-850d-0a2a8784517d 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Attach volume completed successfully. >2018-08-22 07:07:13.553 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-240eb5ab-6063-46d7-a799-f4a2dac55b79 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Unable to get remote copy information for volume volume-1e97788b-88c6-4f54-bc61-925fab5e608d >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova [req-240eb5ab-6063-46d7-a799-f4a2dac55b79 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Failed to notify nova on events: [{'tag': '1e97788b-88c6-4f54-bc61-925fab5e608d', 'name': 'volume-extended', 'server_uuid': '5d59aab1-9561-43a4-b51d-15bca6b36a3f'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-76714a42-b020-45a5-bf75-93c7a9630575) >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova return_raw=True) >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-76714a42-b020-45a5-bf75-93c7a9630575) >2018-08-22 07:07:14.204 39124 ERROR cinder.compute.nova >2018-08-22 07:07:14.227 39124 INFO cinder.volume.manager [req-240eb5ab-6063-46d7-a799-f4a2dac55b79 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Extend volume completed successfully. >2018-08-22 07:12:26.021 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-327bb357-3f5b-4309-85d9-fb0cd28973ea 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:12:26.304 39124 INFO cinder.volume.manager [req-327bb357-3f5b-4309-85d9-fb0cd28973ea 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Terminate volume connection completed successfully. >2018-08-22 07:12:26.447 39124 INFO cinder.volume.manager [req-1a1b9058-149b-4007-8dea-ad1b8fe9fde1 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Detaching volume 1e97788b-88c6-4f54-bc61-925fab5e608d from instance 5d59aab1-9561-43a4-b51d-15bca6b36a3f. >2018-08-22 07:12:26.533 39124 INFO cinder.volume.manager [req-1a1b9058-149b-4007-8dea-ad1b8fe9fde1 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Detach volume completed successfully. >2018-08-22 07:12:30.692 39124 INFO cinder.volume.manager [req-afe6cf6c-cc70-4445-b38d-7fe8f2ae8415 1834b5ef991c4895a983be400966c02f 10fb9a81a3834164a204e7cdbb030f07 - default default] Deleted volume successfully. >2018-08-22 07:12:36.558 39124 INFO cinder.volume.flows.manager.create_volume [req-b69f872c-b8dc-421a-9f63-1a8d06e880ac 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] Volume b9e9bb71-da63-4398-b934-f0e1767cd5b0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b9e9bb71-da63-4398-b934-f0e1767cd5b0'} >2018-08-22 07:12:37.101 39124 INFO cinder.volume.flows.manager.create_volume [req-b69f872c-b8dc-421a-9f63-1a8d06e880ac 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] Volume volume-b9e9bb71-da63-4398-b934-f0e1767cd5b0 (b9e9bb71-da63-4398-b934-f0e1767cd5b0): created successfully >2018-08-22 07:12:37.105 39124 INFO cinder.volume.manager [req-b69f872c-b8dc-421a-9f63-1a8d06e880ac 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] Created volume successfully. >2018-08-22 07:12:38.815 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3b20004b-0e8b-4335-8813-949e48718e8e 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] Unable to get remote copy information for volume volume-b9e9bb71-da63-4398-b934-f0e1767cd5b0 >2018-08-22 07:12:39.133 39124 INFO cinder.volume.manager [req-3b20004b-0e8b-4335-8813-949e48718e8e 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] Extend volume completed successfully. >2018-08-22 07:12:41.346 39124 INFO cinder.volume.manager [req-cf07dbf6-e5eb-4ee1-8848-29749d2e1955 5db1e88791c543da9ef07546a6b48e00 2b771723005f4a44a6a1ee5ce68cc31b - default default] Deleted volume successfully. >2018-08-22 07:12:46.131 39124 INFO cinder.volume.flows.manager.create_volume [req-f3ef05eb-878f-4a4e-b90c-e4ac053c9c2a 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume 85657706-2e45-44b4-95e2-019157b90294: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-85657706-2e45-44b4-95e2-019157b90294'} >2018-08-22 07:12:46.658 39124 INFO cinder.volume.flows.manager.create_volume [req-f3ef05eb-878f-4a4e-b90c-e4ac053c9c2a 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume volume-85657706-2e45-44b4-95e2-019157b90294 (85657706-2e45-44b4-95e2-019157b90294): created successfully >2018-08-22 07:12:46.662 39124 INFO cinder.volume.manager [req-f3ef05eb-878f-4a4e-b90c-e4ac053c9c2a 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Created volume successfully. >2018-08-22 07:12:49.086 39124 INFO cinder.volume.flows.manager.create_volume [req-4ab01d6e-625e-4afa-9540-782b623c7764 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume cdc43943-cfcc-4150-9aae-3b64e885110a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cdc43943-cfcc-4150-9aae-3b64e885110a'} >2018-08-22 07:12:49.630 39124 INFO cinder.volume.flows.manager.create_volume [req-4ab01d6e-625e-4afa-9540-782b623c7764 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume volume-cdc43943-cfcc-4150-9aae-3b64e885110a (cdc43943-cfcc-4150-9aae-3b64e885110a): created successfully >2018-08-22 07:12:49.633 39124 INFO cinder.volume.manager [req-4ab01d6e-625e-4afa-9540-782b623c7764 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Created volume successfully. >2018-08-22 07:12:52.156 39124 INFO cinder.volume.manager [req-22e971a1-7039-4cd1-959d-dae1e6b123e2 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Deleted volume successfully. >2018-08-22 07:12:53.423 39124 INFO cinder.volume.manager [req-d3e85967-fa5b-4595-82cb-20323a462e5d 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Deleted volume successfully. >2018-08-22 07:12:53.985 39124 INFO cinder.volume.flows.manager.create_volume [req-ffb54bd3-71a6-49da-af27-49a3efdbe302 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume c1d0ae29-3cd5-412b-9d84-e39a05ab402d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c1d0ae29-3cd5-412b-9d84-e39a05ab402d'} >2018-08-22 07:12:54.513 39124 INFO cinder.volume.flows.manager.create_volume [req-ffb54bd3-71a6-49da-af27-49a3efdbe302 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume volume-c1d0ae29-3cd5-412b-9d84-e39a05ab402d (c1d0ae29-3cd5-412b-9d84-e39a05ab402d): created successfully >2018-08-22 07:12:54.516 39124 INFO cinder.volume.manager [req-ffb54bd3-71a6-49da-af27-49a3efdbe302 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Created volume successfully. >2018-08-22 07:12:55.394 39124 INFO cinder.volume.flows.manager.create_volume [req-e1c536d8-6a5e-44e4-b39d-de9b32cf2723 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume 20ac473a-bea8-47c2-99cb-2c4471ecd0c0: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-20ac473a-bea8-47c2-99cb-2c4471ecd0c0', 'source_volid': 'c1d0ae29-3cd5-412b-9d84-e39a05ab402d', 'source_volstatus': u'available'} >2018-08-22 07:12:56.848 39124 INFO cinder.volume.flows.manager.create_volume [req-e1c536d8-6a5e-44e4-b39d-de9b32cf2723 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume volume-20ac473a-bea8-47c2-99cb-2c4471ecd0c0 (20ac473a-bea8-47c2-99cb-2c4471ecd0c0): created successfully >2018-08-22 07:12:56.852 39124 INFO cinder.volume.manager [req-e1c536d8-6a5e-44e4-b39d-de9b32cf2723 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Created volume successfully. >2018-08-22 07:12:58.196 39124 INFO cinder.volume.flows.manager.create_volume [req-d99573c0-6656-49d0-b279-c35812309b9b 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume d87b2b5e-73ee-4bdf-9089-f4edb6acfd0d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d87b2b5e-73ee-4bdf-9089-f4edb6acfd0d'} >2018-08-22 07:12:58.726 39124 INFO cinder.volume.flows.manager.create_volume [req-d99573c0-6656-49d0-b279-c35812309b9b 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume volume-d87b2b5e-73ee-4bdf-9089-f4edb6acfd0d (d87b2b5e-73ee-4bdf-9089-f4edb6acfd0d): created successfully >2018-08-22 07:12:58.729 39124 INFO cinder.volume.manager [req-d99573c0-6656-49d0-b279-c35812309b9b 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Created volume successfully. >2018-08-22 07:13:00.494 39124 INFO cinder.volume.manager [req-416c6e42-8d69-4f26-8996-8e3e8a882dd3 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Deleted volume successfully. >2018-08-22 07:13:04.704 39124 INFO cinder.volume.manager [req-a8d570d8-333a-4c66-b19f-2b52e68624de 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Deleted volume successfully. >2018-08-22 07:13:06.348 39124 INFO cinder.volume.flows.manager.create_volume [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume ceb3aa21-dca5-4db6-b88f-4adf98256b98: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-ceb3aa21-dca5-4db6-b88f-4adf98256b98', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b1d4df90>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 07:13:07.240 39124 INFO cinder.image.image_utils [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 07:13:16.909 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] initialize_connection: Did not find a preferred node for volume volume-ceb3aa21-dca5-4db6-b88f-4adf98256b98. >2018-08-22 07:13:17.548 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 07:13:22.410 39124 INFO cinder.image.image_utils [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 07:13:25.015 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:13:25.569 39124 INFO cinder.volume.flows.manager.create_volume [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume volume-ceb3aa21-dca5-4db6-b88f-4adf98256b98 (ceb3aa21-dca5-4db6-b88f-4adf98256b98): created successfully >2018-08-22 07:13:25.573 39124 INFO cinder.volume.manager [req-c2771f15-74cc-42ab-b60f-f243614078a6 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Created volume successfully. >2018-08-22 07:13:26.999 39124 INFO cinder.volume.flows.manager.create_volume [req-2a031e79-c7da-451d-ad14-55686b6f411c 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume 1341a59a-28b1-4c7b-9290-92784af8a497: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1341a59a-28b1-4c7b-9290-92784af8a497'} >2018-08-22 07:13:27.952 39124 INFO cinder.volume.flows.manager.create_volume [req-2a031e79-c7da-451d-ad14-55686b6f411c 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Volume volume-1341a59a-28b1-4c7b-9290-92784af8a497 (1341a59a-28b1-4c7b-9290-92784af8a497): created successfully >2018-08-22 07:13:27.956 39124 INFO cinder.volume.manager [req-2a031e79-c7da-451d-ad14-55686b6f411c 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Created volume successfully. >2018-08-22 07:13:30.124 39124 INFO cinder.volume.manager [req-ff562622-19d3-4808-a4cc-e2c871a4dc6f 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Deleted volume successfully. >2018-08-22 07:13:32.314 39124 INFO cinder.volume.manager [req-4c65add8-e4ce-4d98-8746-099a895f3423 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Deleted volume successfully. >2018-08-22 07:13:34.836 39124 INFO cinder.volume.manager [req-f2d04751-57dd-4c7f-85f0-bd08f51884cf 81eeced24ec049f4a541ecef2824f56d c388f12aacfa4e8ea165cfc256dc98c5 - default default] Deleted volume successfully. >2018-08-22 07:13:44.105 39124 INFO cinder.volume.flows.manager.create_volume [req-b1420720-92b7-41e9-8e28-3414f27cd3f0 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Volume 846bd2d5-e5a5-47b8-9365-5e23a3c1fddf: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-846bd2d5-e5a5-47b8-9365-5e23a3c1fddf'} >2018-08-22 07:13:45.131 39124 INFO cinder.volume.flows.manager.create_volume [req-b1420720-92b7-41e9-8e28-3414f27cd3f0 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Volume volume-846bd2d5-e5a5-47b8-9365-5e23a3c1fddf (846bd2d5-e5a5-47b8-9365-5e23a3c1fddf): created successfully >2018-08-22 07:13:45.135 39124 INFO cinder.volume.manager [req-b1420720-92b7-41e9-8e28-3414f27cd3f0 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Created volume successfully. >2018-08-22 07:13:46.516 39124 INFO cinder.volume.flows.manager.create_volume [req-081bde74-f34f-4029-a895-a09769f12035 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Volume f4487757-7b66-4268-b588-d4b66775252a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f4487757-7b66-4268-b588-d4b66775252a'} >2018-08-22 07:13:47.489 39124 INFO cinder.volume.flows.manager.create_volume [req-081bde74-f34f-4029-a895-a09769f12035 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Volume volume-f4487757-7b66-4268-b588-d4b66775252a (f4487757-7b66-4268-b588-d4b66775252a): created successfully >2018-08-22 07:13:47.493 39124 INFO cinder.volume.manager [req-081bde74-f34f-4029-a895-a09769f12035 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Created volume successfully. >2018-08-22 07:13:48.455 39124 INFO cinder.volume.flows.manager.create_volume [req-75793425-c768-4927-91d2-f8f6c23c7d62 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Volume 3983b621-64b4-43f8-ab2d-10f7a2f7298e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3983b621-64b4-43f8-ab2d-10f7a2f7298e'} >2018-08-22 07:13:49.422 39124 INFO cinder.volume.flows.manager.create_volume [req-75793425-c768-4927-91d2-f8f6c23c7d62 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Volume volume-3983b621-64b4-43f8-ab2d-10f7a2f7298e (3983b621-64b4-43f8-ab2d-10f7a2f7298e): created successfully >2018-08-22 07:13:49.425 39124 INFO cinder.volume.manager [req-75793425-c768-4927-91d2-f8f6c23c7d62 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Created volume successfully. >2018-08-22 07:13:55.442 39124 INFO cinder.volume.manager [req-e2b789c1-e264-4782-9650-21acd59480d5 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Deleted volume successfully. >2018-08-22 07:13:55.666 39124 INFO cinder.volume.manager [req-e692b74f-5b18-4a3d-9424-74a553a62c6f 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Deleted volume successfully. >2018-08-22 07:13:55.970 39124 INFO cinder.volume.manager [req-e23c643a-94b1-4063-8c22-e5ed837a44b0 2b18a67ad8c04e9bbc1d05386aae3060 0ba0268c5bcd461bb9d044c70e7d42f2 - default default] Deleted volume successfully. >2018-08-22 07:14:00.188 39124 INFO cinder.volume.flows.manager.create_volume [req-eba71253-27b9-4610-b184-bf1ed5d88f77 0846c7ed78e944099ed21a62a07434db 57941ecccb484f6ea34dc671d512f19e - default default] Volume 63c44f31-74cd-49a6-8b14-97c98a3dc91e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-63c44f31-74cd-49a6-8b14-97c98a3dc91e'} >2018-08-22 07:14:01.352 39124 INFO cinder.volume.flows.manager.create_volume [req-eba71253-27b9-4610-b184-bf1ed5d88f77 0846c7ed78e944099ed21a62a07434db 57941ecccb484f6ea34dc671d512f19e - default default] Volume volume-63c44f31-74cd-49a6-8b14-97c98a3dc91e (63c44f31-74cd-49a6-8b14-97c98a3dc91e): created successfully >2018-08-22 07:14:01.356 39124 INFO cinder.volume.manager [req-eba71253-27b9-4610-b184-bf1ed5d88f77 0846c7ed78e944099ed21a62a07434db 57941ecccb484f6ea34dc671d512f19e - default default] Created volume successfully. >2018-08-22 07:14:20.772 39124 INFO cinder.volume.manager [req-64d1d6a6-9457-4481-b134-9c84cb4cf6e8 0846c7ed78e944099ed21a62a07434db 57941ecccb484f6ea34dc671d512f19e - default default] Deleted volume successfully. >2018-08-22 07:14:25.119 39124 INFO cinder.volume.flows.manager.create_volume [req-245e5119-735c-49db-adf0-2927eca9b5d5 1f64b0b8a81544f196a0f6497593ea0d c135b9063bab48bdaa1df4de1ace973e - default default] Volume f25c5e37-068c-40f7-b3cc-a8e057b980be: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f25c5e37-068c-40f7-b3cc-a8e057b980be'} >2018-08-22 07:14:25.650 39124 INFO cinder.volume.flows.manager.create_volume [req-245e5119-735c-49db-adf0-2927eca9b5d5 1f64b0b8a81544f196a0f6497593ea0d c135b9063bab48bdaa1df4de1ace973e - default default] Volume volume-f25c5e37-068c-40f7-b3cc-a8e057b980be (f25c5e37-068c-40f7-b3cc-a8e057b980be): created successfully >2018-08-22 07:14:25.653 39124 INFO cinder.volume.manager [req-245e5119-735c-49db-adf0-2927eca9b5d5 1f64b0b8a81544f196a0f6497593ea0d c135b9063bab48bdaa1df4de1ace973e - default default] Created volume successfully. >2018-08-22 07:14:28.512 39124 INFO cinder.volume.manager [req-4f7c2825-bd57-422b-8c44-2f916ae73c5d 1f64b0b8a81544f196a0f6497593ea0d c135b9063bab48bdaa1df4de1ace973e - default default] Deleted volume successfully. >2018-08-22 07:15:16.425 39124 INFO cinder.volume.flows.manager.create_volume [req-50779ec2-c8f1-4bc1-83c1-2bdec7ef15c1 6f403ab192e8489597456e29da869e33 da9c22ca7403412b87a2cc8a566ac563 - default default] Volume 0e2f717c-6801-4aec-9c9a-ed333ddcd862: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0e2f717c-6801-4aec-9c9a-ed333ddcd862'} >2018-08-22 07:15:16.975 39124 INFO cinder.volume.flows.manager.create_volume [req-50779ec2-c8f1-4bc1-83c1-2bdec7ef15c1 6f403ab192e8489597456e29da869e33 da9c22ca7403412b87a2cc8a566ac563 - default default] Volume volume-0e2f717c-6801-4aec-9c9a-ed333ddcd862 (0e2f717c-6801-4aec-9c9a-ed333ddcd862): created successfully >2018-08-22 07:15:16.979 39124 INFO cinder.volume.manager [req-50779ec2-c8f1-4bc1-83c1-2bdec7ef15c1 6f403ab192e8489597456e29da869e33 da9c22ca7403412b87a2cc8a566ac563 - default default] Created volume successfully. >2018-08-22 07:15:20.009 39124 INFO cinder.volume.manager [req-9286c538-c3f4-4fa8-b5e6-7bac702148c7 6f403ab192e8489597456e29da869e33 da9c22ca7403412b87a2cc8a566ac563 - default default] Deleted volume successfully. >2018-08-22 07:15:25.489 39124 INFO cinder.volume.flows.manager.create_volume [req-850e851c-13aa-4ba8-98b4-5093121b3bbe 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Volume c330fa35-2e05-454c-9336-45159e68f70f: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-c330fa35-2e05-454c-9336-45159e68f70f', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b20a89d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 07:15:26.282 39124 INFO cinder.image.image_utils [req-850e851c-13aa-4ba8-98b4-5093121b3bbe 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 07:15:39.614 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 07:15:44.616 39124 INFO cinder.image.image_utils [req-850e851c-13aa-4ba8-98b4-5093121b3bbe 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 07:15:46.511 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-850e851c-13aa-4ba8-98b4-5093121b3bbe 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:15:46.855 39124 INFO cinder.volume.flows.manager.create_volume [req-850e851c-13aa-4ba8-98b4-5093121b3bbe 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Volume volume-c330fa35-2e05-454c-9336-45159e68f70f (c330fa35-2e05-454c-9336-45159e68f70f): created successfully >2018-08-22 07:15:46.859 39124 INFO cinder.volume.manager [req-850e851c-13aa-4ba8-98b4-5093121b3bbe 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Created volume successfully. >2018-08-22 07:15:47.649 39124 INFO cinder.volume.flows.manager.create_volume [req-2faa6dc3-4b46-4542-9f9f-0e8b0e7f66dc 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Volume 83a3b8fb-cf23-4bf8-a489-ef3210145f0e: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-83a3b8fb-cf23-4bf8-a489-ef3210145f0e', 'source_volid': 'c330fa35-2e05-454c-9336-45159e68f70f', 'source_volstatus': u'available'} >2018-08-22 07:15:49.125 39124 INFO cinder.volume.flows.manager.create_volume [req-2faa6dc3-4b46-4542-9f9f-0e8b0e7f66dc 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Volume volume-83a3b8fb-cf23-4bf8-a489-ef3210145f0e (83a3b8fb-cf23-4bf8-a489-ef3210145f0e): created successfully >2018-08-22 07:15:49.128 39124 INFO cinder.volume.manager [req-2faa6dc3-4b46-4542-9f9f-0e8b0e7f66dc 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Created volume successfully. >2018-08-22 07:15:53.391 39124 INFO cinder.volume.manager [req-f8d6bb5a-526e-4a20-8499-f75d1778d44b 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Deleted volume successfully. >2018-08-22 07:15:54.810 39124 INFO cinder.volume.manager [req-2be47b10-e9c2-41e8-801a-f8a18d00f9b1 9ce96b34ac75458bb2ab85de91539da5 d392572773db4631a33f3810df5088c1 - default default] Deleted volume successfully. >2018-08-22 07:15:59.639 39124 INFO cinder.volume.flows.manager.create_volume [req-a14531be-645b-41d8-a403-7057110f1017 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Volume c071b0e6-087a-4edb-b350-9d5aef9bd3c5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c071b0e6-087a-4edb-b350-9d5aef9bd3c5'} >2018-08-22 07:16:00.168 39124 INFO cinder.volume.flows.manager.create_volume [req-a14531be-645b-41d8-a403-7057110f1017 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Volume volume-c071b0e6-087a-4edb-b350-9d5aef9bd3c5 (c071b0e6-087a-4edb-b350-9d5aef9bd3c5): created successfully >2018-08-22 07:16:00.171 39124 INFO cinder.volume.manager [req-a14531be-645b-41d8-a403-7057110f1017 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Created volume successfully. >2018-08-22 07:16:01.928 39124 INFO cinder.volume.flows.manager.create_volume [req-9612a7a2-5494-4565-99c7-3c2a2497a36a 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Volume 47eb790e-c253-4e6a-9f47-d26aa086faab: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-47eb790e-c253-4e6a-9f47-d26aa086faab'} >2018-08-22 07:16:04.580 39124 INFO cinder.volume.flows.manager.create_volume [req-9612a7a2-5494-4565-99c7-3c2a2497a36a 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Volume volume-47eb790e-c253-4e6a-9f47-d26aa086faab (47eb790e-c253-4e6a-9f47-d26aa086faab): created successfully >2018-08-22 07:16:04.585 39124 INFO cinder.volume.manager [req-9612a7a2-5494-4565-99c7-3c2a2497a36a 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Created volume successfully. >2018-08-22 07:16:06.426 39116 INFO cinder.volume.flows.manager.create_volume [req-ff27a784-930a-4c4a-9b00-e236e6be26ec 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Volume 7a827015-a243-454f-aaee-0970656ef5a8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7a827015-a243-454f-aaee-0970656ef5a8'} >2018-08-22 07:16:06.959 39116 INFO cinder.volume.flows.manager.create_volume [req-ff27a784-930a-4c4a-9b00-e236e6be26ec 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Volume volume-7a827015-a243-454f-aaee-0970656ef5a8 (7a827015-a243-454f-aaee-0970656ef5a8): created successfully >2018-08-22 07:16:06.963 39116 INFO cinder.volume.manager [req-ff27a784-930a-4c4a-9b00-e236e6be26ec 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Created volume successfully. >2018-08-22 07:16:10.601 39124 INFO cinder.volume.manager [req-abb46f83-fd49-4bf1-8c3d-d7fc63d0405c 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Deleted volume successfully. >2018-08-22 07:16:10.628 39124 INFO cinder.volume.manager [req-df38b49e-e0f8-4990-b2cf-3045f947efb8 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Deleted volume successfully. >2018-08-22 07:16:11.040 39116 INFO cinder.volume.manager [req-34c47792-31b0-459a-9f63-35bec59e00cf 69ad08b1ca554566816b6c25d6d257a9 75e77837cb7240829a84c5539f146e78 - default default] Deleted volume successfully. >2018-08-22 07:16:31.355 39116 INFO cinder.volume.flows.manager.create_volume [req-850f8107-94de-4a71-ab61-1b6d11dc0c60 d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Volume b69ef198-891b-4a2d-ab9d-87e6180082a3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b69ef198-891b-4a2d-ab9d-87e6180082a3'} >2018-08-22 07:16:31.954 39116 INFO cinder.volume.flows.manager.create_volume [req-850f8107-94de-4a71-ab61-1b6d11dc0c60 d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Volume volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 (b69ef198-891b-4a2d-ab9d-87e6180082a3): created successfully >2018-08-22 07:16:31.958 39116 INFO cinder.volume.manager [req-850f8107-94de-4a71-ab61-1b6d11dc0c60 d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Created volume successfully. >2018-08-22 07:16:33.544 39124 INFO cinder.volume.flows.manager.create_volume [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Volume 04ac0ba0-b313-4855-8fce-6e3e81bbd579: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579'} >2018-08-22 07:16:34.110 39124 INFO cinder.volume.flows.manager.create_volume [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Volume volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579 (04ac0ba0-b313-4855-8fce-6e3e81bbd579): created successfully >2018-08-22 07:16:34.113 39124 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Created volume successfully. >2018-08-22 07:16:41.295 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] initialize_connection: Did not find a preferred node for volume volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579. >2018-08-22 07:16:41.628 39124 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Initialize volume connection completed successfully. >2018-08-22 07:16:42.017 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 07:16:45.661 39116 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Initialize volume connection completed successfully. >2018-08-22 07:16:46.046 39116 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 07:16:51.380 39116 INFO cinder.volume.utils [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Volume copy 1024.00 MB at 439.07 MB/s >2018-08-22 07:16:54.674 39124 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Terminate volume connection completed successfully. >2018-08-22 07:16:54.718 39124 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Remove volume export completed successfully. >2018-08-22 07:16:56.565 39116 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:16:56.840 39116 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Terminate volume connection completed successfully. >2018-08-22 07:16:57.025 39116 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Remove volume export completed successfully. >2018-08-22 07:16:57.337 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 07:17:01.889 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Error running SSH command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 07:17:01.891 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 07:17:01.891 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Error running SSH command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 07:17:01.892 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-b69ef198-891b-4a2d-ab9d-87e6180082a3', '"volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-b69ef198-891b-4a2d-ab9d-87e6180082a3 "volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-22 07:17:01.893 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Unable to rename the logical volume for volume: b69ef198-891b-4a2d-ab9d-87e6180082a3: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-b69ef198-891b-4a2d-ab9d-87e6180082a3', '"volume-04ac0ba0-b313-4855-8fce-6e3e81bbd579"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-22 07:17:02.105 39116 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Complete-Migrate volume completed successfully. >2018-08-22 07:17:02.107 39116 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Migrate volume completed successfully. >2018-08-22 07:17:02.621 39116 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Retype volume completed successfully. >2018-08-22 07:17:03.268 39116 INFO cinder.volume.manager [req-9e9e9b1d-d198-498c-9bf0-d4f4b48dcf8a d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Deleted volume successfully. >2018-08-22 07:17:05.204 39124 WARNING cinder.quota [req-ab11bec7-5f83-4282-a9de-4f231a1eadcc d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-2111846034 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-2111846034, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:17:05.205 39124 WARNING cinder.quota [req-ab11bec7-5f83-4282-a9de-4f231a1eadcc d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-2111846034 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-2111846034, it is now deprecated. Please use the default quota class for default quota. >2018-08-22 07:17:05.458 39124 INFO cinder.volume.manager [req-ab11bec7-5f83-4282-a9de-4f231a1eadcc d6799752368448e0bcc921bbcf01eb8d f60ce54c50764127bf658d55ea7ef5bc - default default] Deleted volume successfully. >2018-08-22 07:17:25.724 39124 INFO cinder.volume.flows.manager.create_volume [req-96b48eb1-25b9-49b3-b79e-536af267f304 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Volume 0c21add5-765e-4c3f-be96-ce156a8bfb94: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0c21add5-765e-4c3f-be96-ce156a8bfb94'} >2018-08-22 07:17:26.556 39124 INFO cinder.volume.flows.manager.create_volume [req-96b48eb1-25b9-49b3-b79e-536af267f304 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Volume volume-0c21add5-765e-4c3f-be96-ce156a8bfb94 (0c21add5-765e-4c3f-be96-ce156a8bfb94): created successfully >2018-08-22 07:17:26.560 39124 INFO cinder.volume.manager [req-96b48eb1-25b9-49b3-b79e-536af267f304 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Created volume successfully. >2018-08-22 07:17:30.174 39124 INFO cinder.volume.manager [req-ee79712b-8f5a-4849-8c42-0a6123817d1e b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Create snapshot completed successfully >2018-08-22 07:17:35.187 39124 INFO cinder.volume.manager [req-a2351dbf-34fa-4f21-9532-b10c4e615ce9 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Create snapshot completed successfully >2018-08-22 07:17:43.235 39124 INFO cinder.volume.manager [req-e0da326b-059e-41fd-8be0-b1367dc11292 431aaf0c357942679e0175cd8337d792 3bf9ac37d8e84ba9a7f0af87c6c231ab - default default] Delete snapshot completed successfully. >2018-08-22 07:17:46.229 39124 INFO cinder.volume.manager [req-45a3bf8b-35a1-48d3-8b0f-a99e06822e79 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Create snapshot completed successfully >2018-08-22 07:17:51.228 39124 INFO cinder.volume.manager [req-31b8bf96-1fcb-4fde-b2a3-39eb9d7f3002 431aaf0c357942679e0175cd8337d792 3bf9ac37d8e84ba9a7f0af87c6c231ab - default default] Delete snapshot completed successfully. >2018-08-22 07:17:55.246 39124 INFO cinder.volume.manager [req-22345871-bd80-46ae-b4eb-7a58db2517d7 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Create snapshot completed successfully >2018-08-22 07:17:59.490 39124 INFO cinder.volume.manager [req-3268c1a1-6e96-43b5-a003-8a0bb06fed68 431aaf0c357942679e0175cd8337d792 3bf9ac37d8e84ba9a7f0af87c6c231ab - default default] Delete snapshot completed successfully. >2018-08-22 07:18:03.187 39124 INFO cinder.volume.manager [req-20cfc060-407e-4a43-8242-c4fc1b57f883 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Create snapshot completed successfully >2018-08-22 07:18:10.033 39124 INFO cinder.volume.manager [req-6a64b66b-8614-4ca1-b6e7-d3f9cb7d8717 431aaf0c357942679e0175cd8337d792 3bf9ac37d8e84ba9a7f0af87c6c231ab - default default] Delete snapshot completed successfully. >2018-08-22 07:18:14.680 39124 INFO cinder.volume.manager [req-85c473e0-8e63-42b0-9c23-56ba64788049 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Delete snapshot completed successfully. >2018-08-22 07:18:16.978 39124 INFO cinder.volume.manager [req-4c4e61ce-f9a3-468c-902b-1e72a303d067 b7da0fa2ee8c42b79dbdfc6522afcbad 8b5723af4e3748209f715f06fb9d74ff - default default] Deleted volume successfully. >2018-08-22 07:18:22.295 39124 INFO cinder.volume.flows.manager.create_volume [req-7ccb8215-b0d4-4053-82ae-806977fd2476 ca514b1a0c0142a99c1c905cd33ab90e c61c50eb960641bf8d104fa9f2b98699 - default default] Volume f8f5c3ba-5be2-4cfe-8a26-385fa015ef88: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f8f5c3ba-5be2-4cfe-8a26-385fa015ef88'} >2018-08-22 07:18:23.441 39124 INFO cinder.volume.flows.manager.create_volume [req-7ccb8215-b0d4-4053-82ae-806977fd2476 ca514b1a0c0142a99c1c905cd33ab90e c61c50eb960641bf8d104fa9f2b98699 - default default] Volume volume-f8f5c3ba-5be2-4cfe-8a26-385fa015ef88 (f8f5c3ba-5be2-4cfe-8a26-385fa015ef88): created successfully >2018-08-22 07:18:23.445 39124 INFO cinder.volume.manager [req-7ccb8215-b0d4-4053-82ae-806977fd2476 ca514b1a0c0142a99c1c905cd33ab90e c61c50eb960641bf8d104fa9f2b98699 - default default] Created volume successfully. >2018-08-22 07:18:27.064 39124 INFO cinder.volume.manager [req-32e8a969-8bf2-4019-8139-9454ad6935e1 ca514b1a0c0142a99c1c905cd33ab90e c61c50eb960641bf8d104fa9f2b98699 - default default] Create snapshot completed successfully >2018-08-22 07:18:31.560 39124 INFO cinder.volume.manager [req-fb815cea-6e57-46da-a01c-8e88dc0f0a3e ca514b1a0c0142a99c1c905cd33ab90e c61c50eb960641bf8d104fa9f2b98699 - default default] Delete snapshot completed successfully. >2018-08-22 07:18:32.976 39124 INFO cinder.volume.manager [req-c4dba799-73ca-4e1b-b535-f32ea9ff80f2 ca514b1a0c0142a99c1c905cd33ab90e c61c50eb960641bf8d104fa9f2b98699 - default default] Deleted volume successfully. >2018-08-22 07:18:36.792 39124 INFO cinder.volume.flows.manager.create_volume [req-c97d874d-79bd-4da9-9430-34add8d3fecc 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Volume 3a62602b-e598-46a2-b998-7d0aadf8e9c3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3a62602b-e598-46a2-b998-7d0aadf8e9c3'} >2018-08-22 07:18:37.318 39124 INFO cinder.volume.flows.manager.create_volume [req-c97d874d-79bd-4da9-9430-34add8d3fecc 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Volume volume-3a62602b-e598-46a2-b998-7d0aadf8e9c3 (3a62602b-e598-46a2-b998-7d0aadf8e9c3): created successfully >2018-08-22 07:18:37.322 39124 INFO cinder.volume.manager [req-c97d874d-79bd-4da9-9430-34add8d3fecc 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Created volume successfully. >2018-08-22 07:18:40.155 39124 INFO cinder.volume.manager [req-76755829-3295-4c2d-a177-215735de35e3 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:18:50.460 39124 INFO cinder.volume.manager [req-d9d7f227-dea6-48fe-a3bc-940f3ba90a2e 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Initialize volume connection completed successfully. >2018-08-22 07:19:12.959 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d9d7f227-dea6-48fe-a3bc-940f3ba90a2e 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:19:13.240 39124 INFO cinder.volume.manager [req-d9d7f227-dea6-48fe-a3bc-940f3ba90a2e 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Terminate volume connection completed successfully. >2018-08-22 07:19:13.284 39124 INFO cinder.volume.manager [req-d9d7f227-dea6-48fe-a3bc-940f3ba90a2e 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Remove volume export completed successfully. >2018-08-22 07:19:14.465 39124 INFO cinder.volume.manager [req-d9d7f227-dea6-48fe-a3bc-940f3ba90a2e 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Deleted volume successfully. >2018-08-22 07:19:33.186 39124 INFO cinder.volume.manager [req-06b50569-8145-4bd9-af87-1f6e16937d0b 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Initialize volume connection completed successfully. >2018-08-22 07:19:37.741 39124 INFO cinder.volume.manager [req-e190c627-70cc-4458-aea9-4d114987375b 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Attaching volume 3a62602b-e598-46a2-b998-7d0aadf8e9c3 to instance 1f318fee-8989-47b7-86a4-3e80a569065a at mountpoint /dev/vdb on host None. >2018-08-22 07:19:37.813 39124 INFO cinder.volume.manager [req-e190c627-70cc-4458-aea9-4d114987375b 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Attach volume completed successfully. >2018-08-22 07:19:40.660 39124 INFO cinder.volume.manager [req-8b23b41b-f58e-4a52-a386-e26cbc7cb06c 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:19:42.706 39124 INFO cinder.volume.manager [req-c572b1b4-1e06-4311-bafc-1ec6a8c25860 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:19:45.236 39124 INFO cinder.volume.manager [req-d70fb483-07bf-4cf4-b9a0-865c66982677 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:19:50.783 39124 INFO cinder.volume.manager [req-acbe71db-f8a0-4a05-a044-4161726a2c14 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:20:00.781 39124 INFO cinder.volume.manager [req-4baa4f77-f0e9-4d33-bd26-f3aff0f81bb7 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:20:03.891 39124 INFO cinder.volume.manager [req-2afe0f14-bb02-4946-b88d-5a6f573044cf 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:20:15.435 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-2b6a294f-5648-40de-beb5-8d2b2f1ff2df 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:20:15.712 39124 INFO cinder.volume.manager [req-2b6a294f-5648-40de-beb5-8d2b2f1ff2df 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Terminate volume connection completed successfully. >2018-08-22 07:20:15.851 39124 INFO cinder.volume.manager [req-160ebf7c-1166-4ac3-a7ee-56cb42547564 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Detaching volume 3a62602b-e598-46a2-b998-7d0aadf8e9c3 from instance 1f318fee-8989-47b7-86a4-3e80a569065a. >2018-08-22 07:20:15.939 39124 INFO cinder.volume.manager [req-160ebf7c-1166-4ac3-a7ee-56cb42547564 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Detach volume completed successfully. >2018-08-22 07:20:20.212 39124 INFO cinder.volume.manager [req-407fec87-9833-434c-8c0a-a032b679ed85 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:20:48.840 39124 INFO cinder.volume.manager [req-76e56d15-1728-4f13-91f5-b771e33dbf5a 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:20:51.285 39124 INFO cinder.volume.manager [req-51f81a37-b0d1-45cd-b6e9-1bca9686f38d 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:21:07.297 39124 INFO cinder.volume.manager [req-550ab9e1-f1e8-40be-a5ff-004d04a63b87 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Initialize volume connection completed successfully. >2018-08-22 07:21:12.007 39124 INFO cinder.volume.manager [req-67a10d74-5749-407c-8b4d-6b2b0dcd0eb6 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Attaching volume 3a62602b-e598-46a2-b998-7d0aadf8e9c3 to instance b2ef62be-ba9a-4a5a-9f61-402067a41939 at mountpoint /dev/vdb on host None. >2018-08-22 07:21:12.082 39124 INFO cinder.volume.manager [req-67a10d74-5749-407c-8b4d-6b2b0dcd0eb6 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Attach volume completed successfully. >2018-08-22 07:21:14.785 39124 INFO cinder.volume.manager [req-7c77e193-5cdf-4c3e-943f-c45bc0b0ddf7 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:21:17.079 39124 INFO cinder.volume.manager [req-74d58bc0-fd58-43bb-93a8-039688b43dd4 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:21:19.523 39124 INFO cinder.volume.manager [req-8a3c07b5-e48f-4bf8-a2df-ee8ac0077780 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:21:22.256 39124 INFO cinder.volume.manager [req-6cc2f12c-ada6-4945-91ba-5ddd64a314a8 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:21:26.562 39124 INFO cinder.volume.manager [req-ea33965e-c86b-4f9c-bfb1-06fa25462a15 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:21:38.905 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-cb4002cc-952e-4eda-a52d-3096529f4c9a 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:21:39.350 39124 INFO cinder.volume.manager [req-cb4002cc-952e-4eda-a52d-3096529f4c9a 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Terminate volume connection completed successfully. >2018-08-22 07:21:39.475 39124 INFO cinder.volume.manager [req-a5f7829b-eb9d-469f-939b-b4df74e884ca 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Detaching volume 3a62602b-e598-46a2-b998-7d0aadf8e9c3 from instance b2ef62be-ba9a-4a5a-9f61-402067a41939. >2018-08-22 07:21:39.561 39124 INFO cinder.volume.manager [req-a5f7829b-eb9d-469f-939b-b4df74e884ca 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Detach volume completed successfully. >2018-08-22 07:21:43.217 39116 INFO cinder.volume.flows.manager.create_volume [req-07627e40-5d1f-425f-b7f0-94eb629ff029 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Volume c8fa243d-b2c7-4c99-a9fe-059e725834b7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c8fa243d-b2c7-4c99-a9fe-059e725834b7'} >2018-08-22 07:21:44.063 39116 INFO cinder.volume.flows.manager.create_volume [req-07627e40-5d1f-425f-b7f0-94eb629ff029 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Volume volume-c8fa243d-b2c7-4c99-a9fe-059e725834b7 (c8fa243d-b2c7-4c99-a9fe-059e725834b7): created successfully >2018-08-22 07:21:44.067 39116 INFO cinder.volume.manager [req-07627e40-5d1f-425f-b7f0-94eb629ff029 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Created volume successfully. >2018-08-22 07:21:46.860 39116 INFO cinder.volume.manager [req-3be1ae48-f396-43f0-9c16-76264c6b9df3 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Create snapshot completed successfully >2018-08-22 07:21:47.911 39116 INFO cinder.volume.flows.manager.create_volume [req-f98a5f93-3383-46d5-89a3-983b25bbca29 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Volume 244f7786-c538-4935-966f-47d3f327ac30: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': '70ef0cb4-9035-45f9-b199-e59a5663fcb0', 'volume_name': 'volume-244f7786-c538-4935-966f-47d3f327ac30'} >2018-08-22 07:22:51.168 39116 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f98a5f93-3383-46d5-89a3-983b25bbca29 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Unable to get remote copy information for volume volume-244f7786-c538-4935-966f-47d3f327ac30 >2018-08-22 07:22:51.698 39116 INFO cinder.volume.flows.manager.create_volume [req-f98a5f93-3383-46d5-89a3-983b25bbca29 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Volume volume-244f7786-c538-4935-966f-47d3f327ac30 (244f7786-c538-4935-966f-47d3f327ac30): created successfully >2018-08-22 07:22:51.703 39116 INFO cinder.volume.manager [req-f98a5f93-3383-46d5-89a3-983b25bbca29 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Created volume successfully. >2018-08-22 07:22:54.736 39116 INFO cinder.volume.manager [req-eda69e6d-49b0-4ba1-bb25-aee0b3fdb177 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Deleted volume successfully. >2018-08-22 07:23:00.252 39124 INFO cinder.volume.manager [req-d4e2902c-e474-4896-9e0e-093a4ff0410c 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:23:01.131 39116 INFO cinder.volume.manager [req-029dc78b-21e1-44b8-98de-b2cb2bddd54c 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Delete snapshot completed successfully. >2018-08-22 07:23:04.738 39116 WARNING cinder.db.sqlalchemy.api [req-0f2a8f4c-967b-41a0-867b-b6fbe7167db1 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-22 07:23:04.773 39124 INFO cinder.volume.manager [req-7f40c214-5301-42de-9991-2ac649b42dca 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Deleted volume successfully. >2018-08-22 07:23:05.165 39116 INFO cinder.volume.manager [req-0f2a8f4c-967b-41a0-867b-b6fbe7167db1 1e852dbf4a5a4517b184f8d95dc9d2a7 be29ac117b2e4f8eb780c62923135874 - default default] Deleted volume successfully. >2018-08-22 07:23:10.320 39124 INFO cinder.volume.flows.manager.create_volume [req-cc6f1e76-07ab-4272-9a03-a70dcb81d22a a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Volume 5275bbe3-7dda-4a5d-88a9-7724e72c827e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5275bbe3-7dda-4a5d-88a9-7724e72c827e'} >2018-08-22 07:23:11.772 39124 INFO cinder.volume.flows.manager.create_volume [req-cc6f1e76-07ab-4272-9a03-a70dcb81d22a a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Volume volume-5275bbe3-7dda-4a5d-88a9-7724e72c827e (5275bbe3-7dda-4a5d-88a9-7724e72c827e): created successfully >2018-08-22 07:23:11.776 39124 INFO cinder.volume.manager [req-cc6f1e76-07ab-4272-9a03-a70dcb81d22a a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Created volume successfully. >2018-08-22 07:23:15.117 39124 INFO cinder.volume.manager [req-6c6669e7-479b-42c7-b7ab-d3a176a1f404 a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Create snapshot completed successfully >2018-08-22 07:23:17.533 39124 INFO cinder.volume.manager [req-3392f468-5abe-434c-b996-f699f312ac12 a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Create snapshot completed successfully >2018-08-22 07:23:20.463 39124 INFO cinder.volume.manager [req-99a3e8da-0900-426b-a456-099369ea206f a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Create snapshot completed successfully >2018-08-22 07:23:26.681 39124 INFO cinder.volume.manager [req-3d3f976c-0b4b-4998-8b81-32e7b2ed02de a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Delete snapshot completed successfully. >2018-08-22 07:23:27.069 39124 INFO cinder.volume.manager [req-74178b15-0b99-44b4-b3f0-49342e034c35 a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Delete snapshot completed successfully. >2018-08-22 07:23:27.442 39124 INFO cinder.volume.manager [req-b0166ca6-c212-4db6-8211-302819cf3277 a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Delete snapshot completed successfully. >2018-08-22 07:23:29.336 39124 INFO cinder.volume.manager [req-33d78e71-d82a-43cd-818a-44b349b5172d a19379045a6d49c88c495113f9678080 8c453ff1e80e457c83237a9078d30773 - default default] Deleted volume successfully. >2018-08-22 07:23:35.319 39124 INFO cinder.volume.flows.manager.create_volume [req-bde597fe-cf3a-4e2e-880d-7a7308656e5e 431b636c3cfe4b21a0a9e646a0f5b6a0 bad3968c72ef4db4b3639bb76f0407f9 - default default] Volume 2e70d767-ff0c-4eb2-b251-ff09a84e1cb2: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-2e70d767-ff0c-4eb2-b251-ff09a84e1cb2'} >2018-08-22 07:23:35.869 39124 INFO cinder.volume.flows.manager.create_volume [req-bde597fe-cf3a-4e2e-880d-7a7308656e5e 431b636c3cfe4b21a0a9e646a0f5b6a0 bad3968c72ef4db4b3639bb76f0407f9 - default default] Volume volume-2e70d767-ff0c-4eb2-b251-ff09a84e1cb2 (2e70d767-ff0c-4eb2-b251-ff09a84e1cb2): created successfully >2018-08-22 07:23:35.873 39124 INFO cinder.volume.manager [req-bde597fe-cf3a-4e2e-880d-7a7308656e5e 431b636c3cfe4b21a0a9e646a0f5b6a0 bad3968c72ef4db4b3639bb76f0407f9 - default default] Created volume successfully. >2018-08-22 07:23:38.493 39124 INFO cinder.volume.manager [req-28643414-5768-4aa5-9c0c-3a7f7bff115d 431b636c3cfe4b21a0a9e646a0f5b6a0 bad3968c72ef4db4b3639bb76f0407f9 - default default] Create snapshot completed successfully >2018-08-22 07:23:41.675 39124 INFO cinder.volume.manager [req-964abbeb-450b-4cc1-abaa-7b4530e49969 431b636c3cfe4b21a0a9e646a0f5b6a0 bad3968c72ef4db4b3639bb76f0407f9 - default default] Delete snapshot completed successfully. >2018-08-22 07:23:43.170 39124 INFO cinder.volume.manager [req-1ff63d6e-f3c9-4905-9fb7-6eb4af9957ee 431b636c3cfe4b21a0a9e646a0f5b6a0 bad3968c72ef4db4b3639bb76f0407f9 - default default] Deleted volume successfully. >2018-08-22 07:24:05.717 39124 INFO cinder.volume.flows.manager.create_volume [req-03c709b3-8e1d-4192-b9ec-3bedcb2f406f 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume dd1ef1d1-12ce-498c-80cd-3cdc2c610051: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-dd1ef1d1-12ce-498c-80cd-3cdc2c610051', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f75b1bb7310>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-22 07:24:06.571 39124 INFO cinder.image.image_utils [req-03c709b3-8e1d-4192-b9ec-3bedcb2f406f 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-22 07:24:16.685 39124 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-22 07:24:21.782 39124 INFO cinder.image.image_utils [req-03c709b3-8e1d-4192-b9ec-3bedcb2f406f 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-22 07:24:23.635 39124 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-03c709b3-8e1d-4192-b9ec-3bedcb2f406f 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Need to remove FC Zone, building initiator target map. >2018-08-22 07:24:24.034 39124 INFO cinder.volume.flows.manager.create_volume [req-03c709b3-8e1d-4192-b9ec-3bedcb2f406f 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume volume-dd1ef1d1-12ce-498c-80cd-3cdc2c610051 (dd1ef1d1-12ce-498c-80cd-3cdc2c610051): created successfully >2018-08-22 07:24:24.037 39124 INFO cinder.volume.manager [req-03c709b3-8e1d-4192-b9ec-3bedcb2f406f 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Created volume successfully. >2018-08-22 07:24:25.514 39124 INFO cinder.volume.flows.manager.create_volume [req-2bbe0ea5-7249-475f-8fd5-c9f122d32040 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume 514c0f3d-f09e-42f7-8424-2dd701cfb0b0: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-514c0f3d-f09e-42f7-8424-2dd701cfb0b0', 'source_volid': 'dd1ef1d1-12ce-498c-80cd-3cdc2c610051', 'source_volstatus': u'available'} >2018-08-22 07:24:27.409 39124 INFO cinder.volume.flows.manager.create_volume [req-2bbe0ea5-7249-475f-8fd5-c9f122d32040 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume volume-514c0f3d-f09e-42f7-8424-2dd701cfb0b0 (514c0f3d-f09e-42f7-8424-2dd701cfb0b0): created successfully >2018-08-22 07:24:27.412 39124 INFO cinder.volume.manager [req-2bbe0ea5-7249-475f-8fd5-c9f122d32040 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Created volume successfully. >2018-08-22 07:24:28.128 39116 INFO cinder.volume.flows.manager.create_volume [req-5030cbfb-aea5-451c-98d4-928d9d650b6b 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume fe120fc3-8312-49d9-9b28-b143ad7cab64: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fe120fc3-8312-49d9-9b28-b143ad7cab64'} >2018-08-22 07:24:28.657 39116 INFO cinder.volume.flows.manager.create_volume [req-5030cbfb-aea5-451c-98d4-928d9d650b6b 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume volume-fe120fc3-8312-49d9-9b28-b143ad7cab64 (fe120fc3-8312-49d9-9b28-b143ad7cab64): created successfully >2018-08-22 07:24:28.661 39116 INFO cinder.volume.manager [req-5030cbfb-aea5-451c-98d4-928d9d650b6b 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Created volume successfully. >2018-08-22 07:24:29.484 39116 INFO cinder.volume.flows.manager.create_volume [req-04d0cf05-e0ff-425b-859d-d51b0f8e0815 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume 7d5d1e6e-b168-4853-93e4-fe31bea23da3: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-7d5d1e6e-b168-4853-93e4-fe31bea23da3', 'source_volid': 'fe120fc3-8312-49d9-9b28-b143ad7cab64', 'source_volstatus': u'available'} >2018-08-22 07:24:52.332 39116 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-04d0cf05-e0ff-425b-859d-d51b0f8e0815 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Unable to get remote copy information for volume volume-7d5d1e6e-b168-4853-93e4-fe31bea23da3 >2018-08-22 07:24:52.602 39116 INFO cinder.volume.flows.manager.create_volume [req-04d0cf05-e0ff-425b-859d-d51b0f8e0815 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Volume volume-7d5d1e6e-b168-4853-93e4-fe31bea23da3 (7d5d1e6e-b168-4853-93e4-fe31bea23da3): created successfully >2018-08-22 07:24:52.606 39116 INFO cinder.volume.manager [req-04d0cf05-e0ff-425b-859d-d51b0f8e0815 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Created volume successfully. >2018-08-22 07:24:56.175 39124 INFO cinder.volume.manager [req-e2b0dc00-5d54-4073-83c3-6b021362b5ee 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Deleted volume successfully. >2018-08-22 07:24:57.668 39124 INFO cinder.volume.manager [req-fdda0e53-da00-43d1-bfa2-32f4526acfcd 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Deleted volume successfully. >2018-08-22 07:24:57.712 39116 INFO cinder.volume.manager [req-df499199-e952-496a-8276-bc51ba0c2683 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Deleted volume successfully. >2018-08-22 07:24:57.944 39116 INFO cinder.volume.manager [req-0cfbcc92-0c1f-4509-909e-f4c04c86e8fc 0d5ce15827044f57b409ec79304673cb 83290a0e42a34ffa9632e26046b6827e - default default] Deleted volume successfully. >2018-08-22 07:25:03.353 39124 INFO cinder.volume.flows.manager.create_volume [req-eb15332e-47ac-4840-b065-8cc70cf5e97b 0cfde42377fd42ffbff76f0d5eea49f7 63a5e29997df45e391ef741373fdbe9e - default default] Volume 3ded0d0e-7273-476d-8bfa-2d70951ae3e3: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-3ded0d0e-7273-476d-8bfa-2d70951ae3e3'} >2018-08-22 07:25:03.883 39124 INFO cinder.volume.flows.manager.create_volume [req-eb15332e-47ac-4840-b065-8cc70cf5e97b 0cfde42377fd42ffbff76f0d5eea49f7 63a5e29997df45e391ef741373fdbe9e - default default] Volume volume-3ded0d0e-7273-476d-8bfa-2d70951ae3e3 (3ded0d0e-7273-476d-8bfa-2d70951ae3e3): created successfully >2018-08-22 07:25:03.886 39124 INFO cinder.volume.manager [req-eb15332e-47ac-4840-b065-8cc70cf5e97b 0cfde42377fd42ffbff76f0d5eea49f7 63a5e29997df45e391ef741373fdbe9e - default default] Created volume successfully. >2018-08-22 07:25:07.343 39124 INFO cinder.volume.manager [req-792a6786-93ab-4806-a5cf-f6b15093c4b8 0cfde42377fd42ffbff76f0d5eea49f7 63a5e29997df45e391ef741373fdbe9e - default default] Deleted volume successfully. >2018-08-22 07:25:31.775 39124 INFO cinder.volume.flows.manager.create_volume [req-086a6dcd-4c9e-44ac-9fef-d370073ca66a eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] Volume 5deded29-ff26-4f75-a856-8eeee732bc28: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5deded29-ff26-4f75-a856-8eeee732bc28'} >2018-08-22 07:25:32.333 39124 INFO cinder.volume.flows.manager.create_volume [req-086a6dcd-4c9e-44ac-9fef-d370073ca66a eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] Volume volume-5deded29-ff26-4f75-a856-8eeee732bc28 (5deded29-ff26-4f75-a856-8eeee732bc28): created successfully >2018-08-22 07:25:32.337 39124 INFO cinder.volume.manager [req-086a6dcd-4c9e-44ac-9fef-d370073ca66a eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] Created volume successfully. >2018-08-22 07:25:34.795 39124 INFO cinder.volume.manager [req-64be2d25-5266-4171-a96e-6f1bddf2ccc3 eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] Deleted volume successfully. >2018-08-22 07:25:35.528 39124 INFO cinder.volume.flows.manager.create_volume [req-a959d500-af0d-4c98-b94f-be4c70ae5812 eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] Volume 85536d2e-500f-41ff-97db-982b2a14c356: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-85536d2e-500f-41ff-97db-982b2a14c356'} >2018-08-22 07:25:36.061 39124 INFO cinder.volume.flows.manager.create_volume [req-a959d500-af0d-4c98-b94f-be4c70ae5812 eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] Volume volume-85536d2e-500f-41ff-97db-982b2a14c356 (85536d2e-500f-41ff-97db-982b2a14c356): created successfully >2018-08-22 07:25:36.064 39124 INFO cinder.volume.manager [req-a959d500-af0d-4c98-b94f-be4c70ae5812 eb7a79f1bb9c451b8df28b6b8ad8b036 1627dd7b553d497194ea2ce59084f3ba - default default] Created volume successfully. >2018-08-22 07:25:37.609 39124 INFO cinder.volume.manager [req-d79897e4-c130-459b-a3d8-12979310fb8c a14c66e36f1e495987a6fc2e2ad63fc2 adca12a0ed4549749825851390393d1e - default default] Transfer volume completed successfully. >2018-08-22 07:25:39.886 39124 INFO cinder.volume.manager [req-06415a86-2be6-4b7f-acf6-3fec951b5c40 43fb849812c94c7aba6a3b8f24f4fc65 096949e0de664bbba435256dafab189c - default default] Deleted volume successfully. >2018-08-22 07:25:50.136 39124 INFO cinder.volume.flows.manager.create_volume [req-b79c8d31-8c75-4953-8858-b7f5751894c4 8992fad903f9421999f7ec137b4ec4b0 4a022058439840b19fa066a9de14607d - default default] Volume 87e07eaa-6c19-4217-8fd1-a09d842d6fa2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-87e07eaa-6c19-4217-8fd1-a09d842d6fa2'} >2018-08-22 07:25:50.666 39124 INFO cinder.volume.flows.manager.create_volume [req-b79c8d31-8c75-4953-8858-b7f5751894c4 8992fad903f9421999f7ec137b4ec4b0 4a022058439840b19fa066a9de14607d - default default] Volume volume-87e07eaa-6c19-4217-8fd1-a09d842d6fa2 (87e07eaa-6c19-4217-8fd1-a09d842d6fa2): created successfully >2018-08-22 07:25:50.670 39124 INFO cinder.volume.manager [req-b79c8d31-8c75-4953-8858-b7f5751894c4 8992fad903f9421999f7ec137b4ec4b0 4a022058439840b19fa066a9de14607d - default default] Created volume successfully. >2018-08-22 07:25:54.368 39124 INFO cinder.volume.manager [req-0adde57d-9c18-47e6-9cdb-ca938e41546f 8992fad903f9421999f7ec137b4ec4b0 4a022058439840b19fa066a9de14607d - default default] Deleted volume successfully. >2018-08-22 07:26:01.503 39124 INFO cinder.volume.flows.manager.create_volume [req-2d62e982-7ae6-4f48-a9f3-78aedd7e9902 7a3ced8404d24087928dcfaa8244df4f d0efb2d4a8584be8b98d3269eeae6429 - default default] Volume c014acdd-33c9-4ace-aa94-6607074dd22c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c014acdd-33c9-4ace-aa94-6607074dd22c'} >2018-08-22 07:26:02.087 39124 INFO cinder.volume.flows.manager.create_volume [req-2d62e982-7ae6-4f48-a9f3-78aedd7e9902 7a3ced8404d24087928dcfaa8244df4f d0efb2d4a8584be8b98d3269eeae6429 - default default] Volume volume-c014acdd-33c9-4ace-aa94-6607074dd22c (c014acdd-33c9-4ace-aa94-6607074dd22c): created successfully >2018-08-22 07:26:02.090 39124 INFO cinder.volume.manager [req-2d62e982-7ae6-4f48-a9f3-78aedd7e9902 7a3ced8404d24087928dcfaa8244df4f d0efb2d4a8584be8b98d3269eeae6429 - default default] Created volume successfully. >2018-08-22 07:26:05.149 39124 INFO cinder.volume.manager [req-16dca61a-6893-4739-b26e-48c7506920b0 7a3ced8404d24087928dcfaa8244df4f d0efb2d4a8584be8b98d3269eeae6429 - default default] Create snapshot completed successfully >2018-08-22 07:26:09.556 39124 INFO cinder.volume.manager [req-aa27214b-17d3-4f46-b907-aae27f795c53 7a3ced8404d24087928dcfaa8244df4f d0efb2d4a8584be8b98d3269eeae6429 - default default] Delete snapshot completed successfully. >2018-08-22 07:26:11.735 39124 INFO cinder.volume.manager [req-1c3fd7cf-9e9a-421f-b9c0-95bc2105746d 7a3ced8404d24087928dcfaa8244df4f d0efb2d4a8584be8b98d3269eeae6429 - default default] Deleted volume successfully. >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 12:23:44.314 39116 ERROR oslo_service.periodic_task >2018-08-22 12:23:53.178 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 12:23:53.178 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 12:23:53.178 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 12:23:53.179 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 12:23:53.179 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task command) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 12:23:53.179 39124 ERROR oslo_service.periodic_task >2018-08-22 12:24:40.936 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 12:24:40.938 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 12:24:40.939 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 12:24:40.939 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 12:24:40.939 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task command) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 12:24:40.940 39116 ERROR oslo_service.periodic_task >2018-08-22 12:24:53.187 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:24:53.261 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:25:10.949 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:25:11.009 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:25:53.185 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:25:53.245 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:26:53.192 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:26:53.275 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:27:53.207 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:27:53.277 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 12:53:32.456 39116 ERROR oslo_service.periodic_task >2018-08-22 12:54:23.212 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 12:54:23.214 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:23.214 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:23.214 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:23.214 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task command) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:23.215 39124 ERROR oslo_service.periodic_task >2018-08-22 12:54:40.952 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 12:54:40.953 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:40.953 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:40.954 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:40.954 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task command) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 12:54:40.954 39116 ERROR oslo_service.periodic_task >2018-08-22 12:54:53.225 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:54:53.299 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:55:10.961 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:55:11.018 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:55:53.230 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:55:53.297 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 12:56:53.229 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 12:56:53.303 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 13:23:52.172 39116 ERROR oslo_service.periodic_task >2018-08-22 13:23:53.274 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 13:23:53.275 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 13:23:53.275 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 13:23:53.275 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 13:23:53.276 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task command) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 13:23:53.276 39124 ERROR oslo_service.periodic_task >2018-08-22 13:24:40.973 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 13:24:40.974 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 13:24:40.974 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 13:24:40.975 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 13:24:40.975 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task command) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 13:24:40.975 39116 ERROR oslo_service.periodic_task >2018-08-22 13:24:53.299 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 13:24:53.390 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 13:25:10.984 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 13:25:11.045 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 13:25:53.282 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 13:25:53.341 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 13:53:37.577 39116 ERROR oslo_service.periodic_task >2018-08-22 13:54:23.298 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 13:54:23.300 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:23.300 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:23.301 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:23.301 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task command) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:23.301 39124 ERROR oslo_service.periodic_task >2018-08-22 13:54:40.988 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 13:54:40.989 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:40.990 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:40.990 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:40.990 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task command) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 13:54:40.991 39116 ERROR oslo_service.periodic_task >2018-08-22 13:54:53.315 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 13:54:53.382 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 13:55:11.000 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 13:55:11.094 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 14:23:58.221 39124 ERROR oslo_service.periodic_task >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 14:23:58.223 39116 ERROR oslo_service.periodic_task >2018-08-22 14:24:41.033 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 14:24:41.034 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 14:24:41.034 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 14:24:41.035 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 14:24:41.035 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task command) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 14:24:41.035 39116 ERROR oslo_service.periodic_task >2018-08-22 14:25:00.345 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 14:25:00.405 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 14:25:11.049 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 14:25:11.113 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 14:53:46.153 39116 ERROR oslo_service.periodic_task >2018-08-22 14:53:53.309 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 14:53:53.310 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 14:53:53.311 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 14:53:53.311 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 14:53:53.311 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task command) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 14:53:53.312 39124 ERROR oslo_service.periodic_task >2018-08-22 14:54:41.085 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 14:54:41.087 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 14:54:41.087 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 14:54:41.088 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 14:54:41.088 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task command) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 14:54:41.088 39116 ERROR oslo_service.periodic_task >2018-08-22 14:54:53.341 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 14:54:53.410 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 14:55:11.099 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 14:55:11.173 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 15:23:32.237 39116 ERROR oslo_service.periodic_task >2018-08-22 15:24:23.341 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 15:24:23.343 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:23.343 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:23.344 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:23.344 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task command) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:23.345 39124 ERROR oslo_service.periodic_task >2018-08-22 15:24:41.057 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 15:24:41.058 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:41.059 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:41.059 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:41.060 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task command) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 15:24:41.060 39116 ERROR oslo_service.periodic_task >2018-08-22 15:24:53.355 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 15:24:53.412 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 15:25:11.072 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 15:25:11.148 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 15:55:36.357 39116 WARNING oslo.service.loopingcall [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 85.32 sec >2018-08-22 15:55:36.377 39124 WARNING oslo.service.loopingcall [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 43.00 sec >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 15:56:37.511 39116 ERROR oslo_service.periodic_task >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 15:56:37.510 39124 ERROR oslo_service.periodic_task >2018-08-22 15:57:37.373 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 15:57:37.390 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 15:57:38.461 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 15:57:38.462 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 16:23:36.371 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.372 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.373 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.373 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.373 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task command) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.374 39116 ERROR oslo_service.periodic_task >2018-08-22 16:23:36.389 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.390 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.390 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.390 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.391 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task command) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:23:36.391 39124 ERROR oslo_service.periodic_task >2018-08-22 16:24:36.393 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 16:24:36.397 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 16:24:36.477 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 16:24:36.479 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 16:55:25.901 39116 ERROR oslo_service.periodic_task >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 16:55:25.903 39124 ERROR oslo_service.periodic_task >2018-08-22 16:55:25.905 39116 WARNING oslo.service.loopingcall [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 49.55 sec >2018-08-22 16:55:25.905 39124 WARNING oslo.service.loopingcall [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 49.50 sec >2018-08-22 16:55:25.913 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.913 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.913 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.913 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.914 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.914 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.914 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.914 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.914 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.914 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task command) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task command) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.915 39116 ERROR oslo_service.periodic_task >task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 16:55:25.915 39124 ERROR oslo_service.periodic_task >2018-08-22 16:56:25.938 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 16:56:25.949 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 16:56:26.024 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 16:56:26.025 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 17:23:40.413 39124 ERROR oslo_service.periodic_task >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 17:23:40.413 39116 ERROR oslo_service.periodic_task >2018-08-22 17:24:40.957 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 17:24:40.984 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 17:24:41.157 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 17:24:41.157 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 17:53:34.353 39124 ERROR oslo_service.periodic_task >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 17:53:34.352 39116 ERROR oslo_service.periodic_task >2018-08-22 17:54:33.007 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 17:54:34.533 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 17:54:40.914 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 17:54:40.975 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 18:24:07.691 39116 ERROR oslo_service.periodic_task >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 18:24:07.690 39124 ERROR oslo_service.periodic_task >2018-08-22 18:24:55.843 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 18:24:55.845 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.846 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.846 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.846 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task command) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.847 39116 ERROR oslo_service.periodic_task >2018-08-22 18:24:55.951 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 18:24:55.952 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.952 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.952 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.953 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task command) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 18:24:55.953 39124 ERROR oslo_service.periodic_task >2018-08-22 18:25:25.898 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 18:25:25.962 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 18:25:25.984 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 18:25:26.021 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 18:53:47.491 39124 ERROR oslo_service.periodic_task >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 18:53:47.490 39116 ERROR oslo_service.periodic_task >2018-08-22 18:54:55.834 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 18:54:55.835 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.836 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.836 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.836 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task command) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.837 39116 ERROR oslo_service.periodic_task >2018-08-22 18:54:55.951 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 18:54:55.952 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.952 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.953 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.953 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task command) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 18:54:55.954 39124 ERROR oslo_service.periodic_task >2018-08-22 18:55:25.858 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 18:55:25.965 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 18:55:25.978 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 18:55:26.063 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 19:24:20.995 39116 ERROR oslo_service.periodic_task >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 19:24:20.996 39124 ERROR oslo_service.periodic_task >2018-08-22 19:24:25.816 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.817 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.817 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.818 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.818 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task command) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.818 39116 ERROR oslo_service.periodic_task >2018-08-22 19:24:25.946 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.947 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.947 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.947 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.948 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task command) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 19:24:25.948 39124 ERROR oslo_service.periodic_task >2018-08-22 19:25:25.825 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 19:25:25.884 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 19:25:25.956 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 19:25:26.018 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 19:53:36.776 39116 ERROR oslo_service.periodic_task >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 19:53:36.779 39124 ERROR oslo_service.periodic_task >2018-08-22 19:54:40.869 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 19:54:40.951 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 19:54:40.993 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 19:54:41.055 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 20:24:12.029 39124 ERROR oslo_service.periodic_task >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 20:24:12.028 39116 ERROR oslo_service.periodic_task >2018-08-22 20:24:55.789 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 20:24:55.790 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.790 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.790 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.791 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task command) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.791 39116 ERROR oslo_service.periodic_task >2018-08-22 20:24:55.988 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 20:24:55.988 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.989 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.989 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.989 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task command) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 20:24:55.990 39124 ERROR oslo_service.periodic_task >2018-08-22 20:25:25.805 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 20:25:25.865 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 20:25:26.003 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 20:25:26.073 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 20:53:39.302 39116 ERROR oslo_service.periodic_task >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 20:53:39.304 39124 ERROR oslo_service.periodic_task >2018-08-22 20:54:40.850 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 20:54:40.945 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 20:54:41.042 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 20:54:41.113 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 21:24:23.581 39124 ERROR oslo_service.periodic_task >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 21:24:23.579 39116 ERROR oslo_service.periodic_task >2018-08-22 21:24:25.794 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:25.794 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:25.795 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:25.795 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:25.795 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task command) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:25.796 39116 ERROR oslo_service.periodic_task >2018-08-22 21:24:26.019 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:26.020 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:26.020 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:26.020 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:26.021 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task command) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-22 21:24:26.021 39124 ERROR oslo_service.periodic_task >2018-08-22 21:25:25.802 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 21:25:25.859 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 21:25:26.051 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 21:25:26.137 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 21:53:40.853 39116 ERROR oslo_service.periodic_task >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 21:53:40.855 39124 ERROR oslo_service.periodic_task >2018-08-22 21:54:40.817 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 21:54:40.880 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 21:54:41.105 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 21:54:41.193 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 22:23:52.058 39116 ERROR oslo_service.periodic_task >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 22:23:52.060 39124 ERROR oslo_service.periodic_task >2018-08-22 22:24:55.809 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 22:24:55.811 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:55.811 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:55.812 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:55.812 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task command) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:55.812 39116 ERROR oslo_service.periodic_task >2018-08-22 22:24:56.020 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 22:24:56.021 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:56.022 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:56.022 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:56.022 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task command) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 22:24:56.023 39124 ERROR oslo_service.periodic_task >2018-08-22 22:25:25.822 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 22:25:25.910 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 22:25:26.031 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 22:25:26.097 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 22:53:50.136 39116 ERROR oslo_service.periodic_task >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 22:53:50.136 39124 ERROR oslo_service.periodic_task >2018-08-22 22:54:55.826 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 22:54:55.828 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:55.828 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:55.829 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:55.829 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task command) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:55.829 39116 ERROR oslo_service.periodic_task >2018-08-22 22:54:56.034 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 22:54:56.034 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:56.035 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:56.035 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:56.035 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task command) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 22:54:56.036 39124 ERROR oslo_service.periodic_task >2018-08-22 22:55:25.841 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 22:55:25.937 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 22:55:26.061 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 22:55:26.135 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 23:23:42.816 39116 ERROR oslo_service.periodic_task >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 23:23:42.817 39124 ERROR oslo_service.periodic_task >2018-08-22 23:24:55.836 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 23:24:55.838 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:55.838 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:55.838 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:55.839 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task command) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:55.839 39116 ERROR oslo_service.periodic_task >2018-08-22 23:24:56.055 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 23:24:56.056 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:56.056 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:56.056 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:56.057 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task command) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 23:24:56.057 39124 ERROR oslo_service.periodic_task >2018-08-22 23:25:25.864 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 23:25:25.932 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 23:25:26.091 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 23:25:26.178 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 23:53:56.574 39124 ERROR oslo_service.periodic_task >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-22 23:53:56.572 39116 ERROR oslo_service.periodic_task >2018-08-22 23:54:55.844 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 23:54:55.846 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:55.846 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:55.846 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:55.847 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task command) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:55.847 39116 ERROR oslo_service.periodic_task >2018-08-22 23:54:56.083 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-22 23:54:56.084 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:56.084 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:56.085 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:56.085 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task command) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-22 23:54:56.085 39124 ERROR oslo_service.periodic_task >2018-08-22 23:55:25.875 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 23:55:25.954 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-22 23:55:26.112 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-22 23:55:26.184 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 00:23:50.253 39116 ERROR oslo_service.periodic_task >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 00:23:50.255 39124 ERROR oslo_service.periodic_task >2018-08-23 00:24:55.884 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 00:24:55.886 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:55.887 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:55.887 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:55.887 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task command) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:55.888 39116 ERROR oslo_service.periodic_task >2018-08-23 00:24:56.104 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 00:24:56.105 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:56.106 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:56.106 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:56.106 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task command) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 00:24:56.107 39124 ERROR oslo_service.periodic_task >2018-08-23 00:25:25.897 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 00:25:25.955 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 00:25:26.133 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 00:25:26.217 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 00:54:03.450 39124 ERROR oslo_service.periodic_task >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 00:54:03.449 39116 ERROR oslo_service.periodic_task >2018-08-23 00:54:55.889 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 00:54:55.892 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:55.892 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:55.893 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:55.893 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task command) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:55.893 39116 ERROR oslo_service.periodic_task >2018-08-23 00:54:56.119 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 00:54:56.120 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:56.120 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:56.121 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:56.121 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task command) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 00:54:56.121 39124 ERROR oslo_service.periodic_task >2018-08-23 00:55:25.912 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 00:55:25.992 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 00:55:26.143 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 00:55:26.224 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 01:23:47.210 39116 ERROR oslo_service.periodic_task >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 01:23:47.212 39124 ERROR oslo_service.periodic_task >2018-08-23 01:24:55.892 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 01:24:55.894 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:55.894 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:55.895 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:55.895 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task command) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:55.895 39116 ERROR oslo_service.periodic_task >2018-08-23 01:24:56.121 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 01:24:56.121 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:56.122 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:56.122 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:56.122 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task command) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 01:24:56.122 39124 ERROR oslo_service.periodic_task >2018-08-23 01:25:25.911 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 01:25:25.984 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 01:25:26.136 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 01:25:26.222 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 01:53:47.552 39116 ERROR oslo_service.periodic_task >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 01:53:47.553 39124 ERROR oslo_service.periodic_task >2018-08-23 01:54:55.925 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 01:54:55.927 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:55.928 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:55.928 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:55.928 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task command) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:55.928 39116 ERROR oslo_service.periodic_task >2018-08-23 01:54:56.125 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 01:54:56.126 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:56.127 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:56.127 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:56.128 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task command) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 01:54:56.128 39124 ERROR oslo_service.periodic_task >2018-08-23 01:55:25.938 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 01:55:25.999 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 01:55:26.154 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 01:55:26.236 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 02:24:16.325 39116 ERROR oslo_service.periodic_task >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 02:24:16.326 39124 ERROR oslo_service.periodic_task >2018-08-23 02:24:25.948 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:25.948 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:25.949 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:25.949 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:25.949 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task command) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:25.950 39116 ERROR oslo_service.periodic_task >2018-08-23 02:24:26.125 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:26.126 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:26.126 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:26.126 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:26.127 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task command) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 02:24:26.127 39124 ERROR oslo_service.periodic_task >2018-08-23 02:25:25.975 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 02:25:26.072 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 02:25:26.134 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 02:25:26.197 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 02:53:47.385 39116 ERROR oslo_service.periodic_task >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 02:53:47.388 39124 ERROR oslo_service.periodic_task >2018-08-23 02:54:55.935 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 02:54:55.937 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:55.938 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:55.938 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:55.938 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task command) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:55.939 39116 ERROR oslo_service.periodic_task >2018-08-23 02:54:56.107 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 02:54:56.108 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:56.108 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:56.109 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:56.109 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task command) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 02:54:56.109 39124 ERROR oslo_service.periodic_task >2018-08-23 02:55:25.941 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 02:55:26.002 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 02:55:26.118 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 02:55:26.185 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 03:24:03.406 39116 ERROR oslo_service.periodic_task >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 03:24:03.407 39124 ERROR oslo_service.periodic_task >2018-08-23 03:24:55.919 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 03:24:55.921 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:55.922 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:55.922 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:55.922 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task command) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:55.923 39116 ERROR oslo_service.periodic_task >2018-08-23 03:24:56.105 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 03:24:56.106 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:56.106 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:56.107 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:56.107 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task command) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 03:24:56.107 39124 ERROR oslo_service.periodic_task >2018-08-23 03:25:25.932 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 03:25:25.990 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 03:25:26.118 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 03:25:26.193 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 03:54:02.980 39116 ERROR oslo_service.periodic_task >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 03:54:02.981 39124 ERROR oslo_service.periodic_task >2018-08-23 03:54:55.921 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 03:54:55.924 39116 ERROR cinder.ssh_utils [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:55.924 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:55.925 39116 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:55.925 39116 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task [req-5ef2d500-9604-4f34-b749-6a3c5e9e71ce - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task command) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:55.925 39116 ERROR oslo_service.periodic_task >2018-08-23 03:54:56.101 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 03:54:56.102 39124 ERROR cinder.ssh_utils [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:56.102 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:56.102 39124 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:56.103 39124 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task [req-0ab1fb84-496e-41f3-9e09-4d785b63010e - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task command) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 03:54:56.103 39124 ERROR oslo_service.periodic_task >2018-08-23 03:55:25.934 39116 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 03:55:25.993 39116 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 03:55:26.113 39124 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 03:55:26.179 39124 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 04:08:51.163 38821 INFO oslo_service.service [req-d7352b2b-8609-48b2-9512-ea3bc03a7f1c - - - - -] Caught SIGTERM, stopping children >2018-08-23 04:08:51.168 38821 INFO oslo_service.service [req-d7352b2b-8609-48b2-9512-ea3bc03a7f1c - - - - -] Waiting on 2 children to exit >2018-08-23 04:08:59.636 39116 WARNING py.warnings [-] /usr/lib/python2.7/site-packages/amqp/connection.py:304: AMQPDeprecationWarning: The .transport attribute on the connection was accessed before >the connection was established. This is supported for now, but will >be deprecated in amqp 2.2.0. > >Since amqp 2.0 you have to explicitly call Connection.connect() >before using the connection. > > W_FORCE_CONNECT.format(attr=attr))) >: error: [Errno 104] Connection reset by peer >2018-08-23 04:08:59.753 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds. Client port: None: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:00.879 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: <AMQPError: unknown error>. Trying again in 1 seconds. Client port: None: RecoverableConnectionError: <AMQPError: unknown error> >2018-08-23 04:09:02.005 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:02.670 39116 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:03.982 39124 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:04.125 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:08.248 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 6 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:09.942 39124 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:09.944 39124 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:14.374 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: <AMQPError: unknown error>. Trying again in 1 seconds. Client port: None: RecoverableConnectionError: <AMQPError: unknown error> >2018-08-23 04:09:14.695 39116 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:15.495 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:17.618 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:17.790 39116 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:19.106 39124 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:21.740 39116 ERROR oslo.messaging._drivers.impl_rabbit [-] [2046dc1b-0a7d-4cb3-87ff-c8008305cfbf] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 6 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:24.959 39124 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 04:09:25.062 39124 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:25.070 39124 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED >2018-08-23 04:09:25.921 38821 INFO oslo_service.service [req-d7352b2b-8609-48b2-9512-ea3bc03a7f1c - - - - -] Child 39116 exited with status 0 >2018-08-23 04:09:26.121 38821 INFO oslo_service.service [req-d7352b2b-8609-48b2-9512-ea3bc03a7f1c - - - - -] Child 39124 exited with status 0 >2018-08-23 04:37:07.723 7483 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 04:37:08.116 7483 WARNING oslo_db.sqlalchemy.engines [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 111] ECONNREFUSED)") >2018-08-23 04:37:18.129 7483 WARNING oslo_db.sqlalchemy.engines [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:28.138 7483 WARNING oslo_db.sqlalchemy.engines [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:38.150 7483 WARNING oslo_db.sqlalchemy.engines [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:48.162 7483 WARNING oslo_db.sqlalchemy.engines [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:37:58.174 7483 WARNING oslo_db.sqlalchemy.engines [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 04:38:08.336 7483 INFO cinder.rpc [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 04:38:08.341 7483 INFO cinder.rpc [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 04:38:08.351 7483 WARNING cinder.keymgr.conf_key_mgr [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-23 04:38:09.636 7483 INFO cinder.volume.manager [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 04:38:09.651 7483 INFO oslo_service.service [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Starting 1 workers >2018-08-23 04:38:09.661 17503 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 04:38:09.676 17503 INFO cinder.volume.manager [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 04:38:10.682 7483 INFO cinder.volume.manager [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 04:38:10.707 7483 INFO oslo_service.service [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Starting 1 workers >2018-08-23 04:38:10.718 7483 WARNING oslo_config.cfg [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 04:38:10.718 17535 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 04:38:10.726 7483 WARNING oslo_config.cfg [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:10.727 7483 WARNING oslo_config.cfg [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:10.728 7483 WARNING oslo_config.cfg [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:10.752 17535 INFO cinder.volume.manager [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 04:38:39.720 17503 ERROR cinder.ssh_utils [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 04:38:39.722 17503 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:39.722 17503 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:39.723 17503 ERROR cinder.volume.manager >2018-08-23 04:38:39.759 17503 WARNING oslo_config.cfg [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:39.759 17503 WARNING oslo_config.cfg [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:39.760 17503 WARNING oslo_config.cfg [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:40.760 17535 ERROR cinder.ssh_utils [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 04:38:40.761 17535 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:40.761 17535 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 04:38:40.762 17535 ERROR cinder.volume.manager >2018-08-23 04:38:40.784 17535 WARNING oslo_config.cfg [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:40.785 17535 WARNING oslo_config.cfg [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:38:40.785 17535 WARNING oslo_config.cfg [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 04:39:39.851 17503 INFO cinder.volume.manager [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 04:39:39.854 17503 ERROR cinder.utils [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 04:39:39.854 17503 ERROR cinder.volume.manager [req-7ce2f380-92d0-4718-b13f-82535926045e - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 04:39:40.856 17535 INFO cinder.volume.manager [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 04:39:40.857 17535 ERROR cinder.utils [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 04:39:40.858 17535 ERROR cinder.volume.manager [req-353a6ccf-6835-49b3-bccc-392bca0753ee - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 04:39:49.856 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:39:50.859 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:39:59.857 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:00.860 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:09.858 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:10.861 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:19.858 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:20.862 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:28.861 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:40:29.861 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:30.862 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:34.860 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:40:39.862 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:40.864 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:49.863 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:50.864 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:40:59.863 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:00.865 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:09.864 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:10.866 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:19.866 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:20.867 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:28.855 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:41:29.868 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:30.868 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:34.862 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:41:39.868 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:40.869 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:49.869 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:50.869 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:41:59.871 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:00.870 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:09.872 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:10.870 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:19.873 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:20.871 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:28.858 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:42:29.874 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:30.872 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:34.864 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:42:39.875 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:40.873 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:49.875 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:50.873 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:42:59.876 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:00.874 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:09.877 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:10.875 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:19.879 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:20.876 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:28.860 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:43:29.882 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:30.877 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:34.866 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:43:39.882 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:40.878 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:49.884 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:50.879 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:43:59.886 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:00.880 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:09.887 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:10.880 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:19.887 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:20.881 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:28.863 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:44:29.889 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:30.881 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:34.867 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:44:39.889 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:40.882 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:49.890 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:50.882 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:44:59.892 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:00.884 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:09.893 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:10.884 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:19.894 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:20.885 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:28.865 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:45:29.896 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:30.886 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:34.870 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:45:39.897 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:40.887 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:49.898 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:50.888 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:45:59.898 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:00.888 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:09.899 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:10.889 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:19.901 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:20.890 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:28.869 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:46:29.904 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:30.891 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:34.871 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:46:39.905 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:40.892 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:49.905 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:50.892 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:46:59.906 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:00.893 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:09.907 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:10.893 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:19.909 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:20.894 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:28.871 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:47:29.910 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:30.897 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:34.873 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:47:39.910 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:40.898 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:49.910 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:50.898 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:47:59.911 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:00.899 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:09.912 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:10.900 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:19.913 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:20.900 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:28.874 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:48:29.915 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:30.901 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:34.874 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:48:39.916 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:40.902 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:49.917 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:50.903 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:48:59.918 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:00.903 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:09.919 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:10.904 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:19.920 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:20.904 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:28.877 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:49:29.921 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:30.905 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:34.876 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:49:39.921 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:40.906 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:49.923 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:50.906 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:49:59.923 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:00.907 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:09.924 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:10.907 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:19.925 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:20.908 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:28.880 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:50:29.926 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:30.909 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:34.878 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:50:39.926 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:40.909 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:49.927 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:50.910 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:50:59.928 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:00.910 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:09.929 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:10.911 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:19.929 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:20.912 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:28.883 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:51:29.930 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:30.913 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:34.881 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:51:39.931 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:40.913 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:49.932 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:50.914 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:51:59.933 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:00.916 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:09.935 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:10.916 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:19.936 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:20.917 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:28.886 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:52:29.936 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:30.918 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:34.883 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:52:39.937 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:40.918 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:49.937 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:50.919 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:52:59.940 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:00.919 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:09.940 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:10.920 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:19.942 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:20.920 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:28.888 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:53:29.943 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:30.921 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:34.885 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:53:39.944 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:40.922 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:49.945 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:50.923 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:53:59.946 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:00.924 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:09.947 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:10.924 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:19.948 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:20.925 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:28.890 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:54:29.948 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:30.925 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:34.886 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:54:39.949 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:40.926 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:49.950 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:50.927 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:54:59.951 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:00.927 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:09.952 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:10.928 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:19.953 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:20.929 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:28.891 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:55:29.953 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:30.929 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:34.888 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:55:39.954 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:40.930 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:49.955 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:50.930 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:55:59.955 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:00.931 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:09.956 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:10.932 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:19.958 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:20.932 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:28.894 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:56:29.959 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:30.933 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:34.889 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:56:39.960 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:40.934 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:49.960 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:50.934 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:56:59.961 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:00.935 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:09.964 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:10.935 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:19.965 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:20.936 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:28.896 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:57:29.966 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:30.937 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:34.891 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:57:39.966 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:40.938 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:49.967 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:50.938 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:57:59.968 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:00.939 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:09.969 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:10.940 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:19.970 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:20.940 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:28.898 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:58:29.971 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:30.941 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:34.892 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:58:39.972 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:40.942 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:49.973 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:50.942 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:58:59.973 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:00.943 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:09.974 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:10.944 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:19.975 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:20.944 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:28.900 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 04:59:29.977 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:30.946 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:34.893 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 04:59:39.977 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:40.947 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:49.978 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:50.947 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 04:59:59.978 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:00.948 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:09.979 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:10.948 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:19.981 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:20.949 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:28.902 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:00:29.982 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:30.950 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:34.895 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:00:39.983 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:40.951 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:49.983 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:50.951 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:00:59.985 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:00.952 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:09.986 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:10.952 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:19.988 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:20.953 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:29.990 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:30.953 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:34.896 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:01:39.991 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:40.954 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:49.991 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:50.955 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:01:59.992 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:00.955 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:09.994 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:10.956 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:19.995 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:20.956 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:28.907 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:02:29.996 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:30.957 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:34.897 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:02:39.996 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:40.958 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:49.997 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:50.959 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:02:59.998 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:00.959 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:09.999 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:10.960 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:20.000 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:20.961 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:28.910 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:03:30.001 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:30.961 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:34.898 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:03:40.002 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:40.962 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:50.002 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:03:50.963 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:00.003 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:00.963 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:10.003 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:10.964 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:20.004 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:20.964 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:28.912 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:04:30.006 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:30.965 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:34.900 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:04:40.007 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:40.966 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:50.008 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:04:50.967 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:00.008 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:00.968 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:10.009 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:10.968 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:20.009 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:20.969 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:28.913 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:05:30.010 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:30.970 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:40.011 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:40.972 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:50.011 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:05:50.972 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:00.012 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:00.973 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:10.013 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:10.974 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:20.014 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:20.975 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:28.915 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:06:30.015 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:30.976 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:34.903 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:06:40.016 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:40.978 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:50.016 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:06:50.978 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:00.017 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:00.979 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:10.018 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:10.980 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:20.018 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:20.980 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:28.918 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:07:30.021 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:30.981 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:34.905 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:07:40.021 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:40.982 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:50.021 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:07:50.983 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:00.022 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:00.983 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:10.023 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:10.984 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:20.024 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:20.985 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:28.920 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:08:30.025 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:30.986 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:34.906 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:08:40.026 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:40.987 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:50.027 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:08:50.988 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:00.028 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:00.988 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:10.028 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:10.989 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:20.029 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:20.989 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:28.916 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:09:30.030 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:30.990 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:34.907 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:09:40.031 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:40.990 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:50.031 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:09:50.991 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:00.032 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:00.994 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:10.032 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:10.995 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:20.033 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:20.995 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:28.918 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:10:30.034 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:30.996 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:34.909 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:10:40.035 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:40.996 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:50.035 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:10:50.997 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:00.036 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:00.998 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:10.036 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:10.999 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:20.039 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:20.999 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:28.922 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:11:30.041 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:31.000 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:34.910 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:11:40.041 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:41.001 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:50.042 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:11:51.002 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:00.042 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:01.002 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:10.043 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:11.003 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:20.044 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:21.003 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:28.923 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:12:30.045 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:31.004 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:34.912 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:12:40.045 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:41.005 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:50.046 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:12:51.005 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:00.046 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:01.006 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:10.047 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:11.007 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:20.048 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:21.008 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:28.925 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:13:30.049 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:31.008 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:34.913 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:13:40.051 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:41.009 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:50.052 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:13:51.009 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:00.052 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:01.010 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:10.053 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:11.011 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:20.054 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:21.011 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:28.927 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:14:30.054 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:31.012 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:34.914 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:14:40.055 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:41.012 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:50.055 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:14:51.013 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:00.056 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:01.014 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:10.056 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:11.015 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:20.058 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:21.016 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:28.929 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:15:30.060 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:31.017 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:34.916 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:15:40.060 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:41.017 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:50.061 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:15:51.018 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:00.061 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:01.019 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:10.063 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:11.020 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:20.065 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:21.021 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:28.932 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:16:30.066 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:31.021 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:34.917 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:16:40.066 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:41.022 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:50.067 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:16:51.023 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:00.068 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:01.023 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:10.069 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:11.024 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:20.072 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:21.025 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:28.934 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:17:30.073 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:31.025 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:34.920 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:17:40.074 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:41.026 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:50.074 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:17:51.027 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:00.075 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:01.027 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:10.076 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:11.028 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:20.076 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:21.028 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:28.936 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:18:30.077 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:31.029 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:34.922 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:18:40.078 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:41.030 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:50.079 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:18:51.030 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:00.080 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:01.032 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:10.081 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:11.032 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:20.082 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:21.033 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:28.938 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:19:30.084 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:31.034 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:34.923 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:19:40.084 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:41.034 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:50.085 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:19:51.035 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:00.086 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:01.036 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:10.086 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:11.036 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:20.089 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:21.037 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:28.941 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:20:30.090 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:31.038 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:34.924 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:20:40.091 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:41.038 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:50.093 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:20:51.039 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:00.093 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:01.040 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:10.094 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:11.041 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:20.094 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:21.041 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:28.943 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:21:30.095 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:31.042 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:34.925 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:21:40.096 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:41.042 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:50.395 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:21:51.342 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:00.397 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:01.343 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:10.398 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:11.343 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:20.399 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:21.345 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:29.246 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:22:30.400 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:31.345 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:35.226 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:22:40.401 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:41.346 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:50.403 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:22:51.346 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:00.405 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:01.347 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:10.407 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:11.348 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:20.408 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:21.349 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:29.249 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:23:30.409 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:31.350 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:35.227 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:23:40.410 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:41.351 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:50.410 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:23:51.351 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:00.411 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:01.352 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:10.412 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:11.354 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:20.413 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:21.355 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:29.252 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:24:30.416 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:31.356 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:35.229 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:24:40.417 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:41.357 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:50.417 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:24:51.358 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:00.419 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:01.359 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:10.421 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:11.359 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:20.421 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:21.360 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:29.254 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:25:30.424 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:31.361 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:35.231 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:25:40.425 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:41.362 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:50.426 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:25:51.363 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:00.427 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:01.364 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:10.429 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:11.365 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:20.431 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:21.365 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:29.257 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:26:30.433 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:31.366 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:35.233 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:26:40.434 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:41.367 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:50.435 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:26:51.367 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:00.436 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:01.368 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:10.437 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:11.369 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:20.437 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:21.370 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:29.259 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:27:30.439 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:31.370 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:35.234 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:27:40.439 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:41.371 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:50.440 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:27:51.372 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:00.441 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:01.373 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:10.441 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:11.374 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:20.442 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:21.375 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:29.262 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:28:30.443 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:31.376 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:35.235 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:28:40.444 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:41.377 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:50.444 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:28:51.377 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:00.445 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:01.378 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:10.446 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:11.379 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:20.447 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:21.380 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:29.263 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:29:30.448 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:31.381 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:35.237 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:29:40.448 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:41.381 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:50.449 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:29:51.382 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:00.451 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:01.383 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:10.452 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:11.383 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:20.453 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:21.384 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:29.267 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:30:30.454 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:31.384 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:35.239 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:30:40.454 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:41.385 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:50.455 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:30:51.385 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:00.455 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:01.386 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:10.460 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:11.386 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:20.461 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:21.387 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:29.269 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:31:30.462 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:31.388 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:35.240 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:31:40.463 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:41.389 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:50.463 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:31:51.389 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:00.465 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:01.390 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:10.466 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:11.391 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:20.467 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:21.392 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:29.272 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:32:30.469 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:31.393 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:35.243 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:32:40.470 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:41.394 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:50.470 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:32:51.395 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:00.471 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:01.395 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:10.472 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:11.396 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:20.472 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:21.397 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:29.275 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:33:30.474 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:31.399 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:35.245 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:33:40.475 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:41.399 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:50.475 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:33:51.400 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:00.476 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:01.401 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:10.477 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:11.401 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:20.477 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:21.402 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:29.278 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:34:30.480 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:31.403 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:35.247 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:34:40.481 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:41.404 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:50.482 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:34:51.404 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:00.483 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:01.405 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:10.484 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:11.406 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:20.485 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:21.406 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:29.280 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:35:30.486 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:31.407 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:35.248 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:35:40.487 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:41.409 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:50.489 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:35:51.409 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:00.490 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:01.410 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:10.492 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:11.410 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:20.493 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:21.412 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:29.283 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:36:30.494 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:31.413 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:35.249 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:36:40.494 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:41.414 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:50.495 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:36:51.415 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:00.495 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:01.415 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:10.496 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:11.416 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:20.496 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:21.417 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:29.286 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:37:30.498 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:31.418 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:35.251 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:37:40.498 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:41.418 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:50.499 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:37:51.419 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:00.499 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:01.420 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:10.500 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:11.421 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:20.502 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:21.421 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:29.288 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:38:30.503 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:31.422 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:35.254 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:38:40.504 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:41.422 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:50.505 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:38:51.423 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:00.505 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:01.424 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:10.506 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:11.424 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:20.507 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:21.425 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:29.292 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:39:30.508 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:31.426 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:35.257 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:39:40.508 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:41.427 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:50.509 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:39:51.428 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:00.511 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:01.429 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:10.511 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:11.430 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:20.512 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:21.430 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:29.296 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:40:30.513 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:31.431 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:35.258 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:40:40.513 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:41.432 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:50.514 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:40:51.432 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:00.514 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:01.433 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:10.516 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:11.433 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:20.517 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:21.434 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:29.299 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:41:30.518 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:31.435 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:35.260 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:41:40.518 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:41.436 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:50.520 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:41:51.437 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:00.521 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:01.438 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:10.522 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:11.439 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:20.522 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:21.440 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:29.304 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:42:30.525 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:31.440 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:35.261 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:42:40.525 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:41.441 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:50.526 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:42:51.441 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:00.527 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:01.442 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:10.528 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:11.443 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:20.530 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:21.444 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:29.306 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:43:30.532 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:31.446 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:35.263 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:43:40.532 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:41.446 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:50.533 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:43:51.447 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:00.534 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:01.448 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:10.535 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:11.449 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:20.535 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:21.449 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:29.309 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:44:30.537 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:31.450 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:35.264 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:44:40.537 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:41.451 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:50.538 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:44:51.451 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:00.539 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:01.452 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:10.539 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:11.453 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:20.540 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:21.453 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:29.312 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:45:30.541 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:31.454 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:35.266 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:45:40.542 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:41.455 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:50.543 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:45:51.456 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:00.544 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:01.456 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:10.547 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:11.457 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:20.548 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:21.458 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:29.315 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:46:30.548 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:31.458 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:35.267 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:46:40.549 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:41.459 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:50.549 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:46:51.459 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:00.550 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:01.460 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:10.552 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:11.461 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:20.553 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:21.461 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:29.318 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:47:30.554 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:31.462 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:35.269 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:47:40.556 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:41.463 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:50.558 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:47:51.464 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:00.559 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:01.465 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:10.560 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:11.465 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:20.561 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:21.466 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:29.321 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:48:30.561 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:31.467 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:35.270 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:48:40.562 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:41.468 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:50.563 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:48:51.469 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:00.564 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:01.469 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:10.564 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:11.470 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:20.565 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:21.471 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:29.324 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:49:30.566 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:31.472 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:35.271 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:49:40.566 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:41.472 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:50.567 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:49:51.474 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:00.567 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:01.474 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:10.568 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:11.474 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:20.568 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:21.475 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:29.327 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:50:30.569 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:31.476 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:35.272 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:50:40.570 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:41.477 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:50.571 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:50:51.477 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:00.571 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:01.478 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:10.572 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:11.479 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:20.573 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:21.480 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:29.330 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:51:30.574 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:31.481 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:35.274 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:51:40.575 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:41.481 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:50.575 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:51:51.482 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:00.577 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:01.483 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:10.579 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:11.483 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:20.581 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:21.484 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:29.333 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:52:30.582 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:31.484 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:35.275 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:52:40.583 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:41.485 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:50.584 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:52:51.486 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:00.585 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:01.486 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:10.586 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:11.487 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:20.588 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:21.487 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:30.589 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:31.488 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:35.277 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:53:40.592 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:41.489 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:50.593 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:53:51.489 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:00.595 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:01.490 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:10.595 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:11.491 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:20.596 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:21.492 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:29.338 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:54:30.598 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:31.494 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:35.278 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:54:40.599 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:41.495 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:50.600 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:54:51.496 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:00.603 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:01.496 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:10.604 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:11.498 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:20.605 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:21.499 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:29.341 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:55:30.607 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:31.500 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:35.280 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:55:40.608 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:41.500 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:50.608 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:55:51.501 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:00.610 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:01.502 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:10.612 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:11.503 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:20.614 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:21.504 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:29.344 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:56:30.615 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:31.505 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:35.281 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:56:40.615 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:41.505 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:50.616 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:56:51.506 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:00.617 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:01.507 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:10.620 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:11.507 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:20.621 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:21.508 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:29.347 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:57:30.623 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:31.509 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:35.284 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 05:57:40.624 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:41.509 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:50.625 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:57:51.510 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:00.627 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:01.511 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:10.628 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:11.511 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:20.630 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:21.513 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:29.350 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:58:30.632 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:31.513 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:40.633 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:41.514 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:50.635 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:58:51.515 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:00.637 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:01.516 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:10.640 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:11.516 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:20.642 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:21.517 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:29.354 17503 WARNING cinder.volume.manager [req-f7d15a1b-146a-4c20-9d40-e6ae3ad0e647 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 05:59:30.644 17503 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:31.518 17535 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 05:59:32.988 7483 INFO oslo_service.service [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Caught SIGTERM, stopping children >2018-08-23 05:59:32.994 7483 INFO oslo_service.service [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Waiting on 2 children to exit >2018-08-23 05:59:35.288 17535 WARNING cinder.volume.manager [req-aaff90bb-6abb-49dd-a218-e79c301c26da - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 06:00:29.375 7483 INFO oslo_service.service [req-6a10bac8-1901-499c-a479-3fd72cf21da2 - - - - -] Child 17503 exited with status 0 >2018-08-23 06:00:33.002 17535 INFO oslo_service.service [-] Parent process has died unexpectedly, exiting >2018-08-23 06:00:35.220 505017 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 06:00:35.674 505017 INFO cinder.rpc [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 06:00:35.680 505017 INFO cinder.rpc [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 06:00:35.691 505017 WARNING cinder.keymgr.conf_key_mgr [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-23 06:00:36.734 505017 INFO cinder.volume.manager [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 06:00:36.750 505017 INFO oslo_service.service [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Starting 1 workers >2018-08-23 06:00:36.758 505464 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 06:00:36.773 505464 INFO cinder.volume.manager [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 06:00:36.788 505464 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:00:36.847 505464 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:00:37.781 505017 INFO cinder.volume.manager [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 06:00:37.803 505017 INFO oslo_service.service [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Starting 1 workers >2018-08-23 06:00:37.812 505523 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 06:00:37.812 505017 WARNING oslo_config.cfg [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 06:00:37.819 505017 WARNING oslo_config.cfg [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:37.820 505017 WARNING oslo_config.cfg [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:37.820 505017 WARNING oslo_config.cfg [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:37.826 505523 INFO cinder.volume.manager [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 06:00:37.868 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:00:37.942 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:00:39.479 505464 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] WWPN on node 155: [u'500507680C1648C2', u'500507680C3648C2', u'500507680C3748C2', u'500507680C3848C2', u'500507680C1348C2', u'500507680C1848C2', u'500507680C3348C2', u'500507680C3548C2', u'500507680C1448C2', u'500507680C1548C2', u'500507680C1148C2', u'500507680C1748C2', u'500507680C1248C2', u'500507680C3248C2', u'500507680C3448C2', u'500507680C3148C2']. >2018-08-23 06:00:40.658 505464 WARNING oslo_config.cfg [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:40.659 505464 WARNING oslo_config.cfg [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:40.659 505464 WARNING oslo_config.cfg [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:40.794 505464 INFO cinder.volume.manager [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Driver initialization completed successfully. >2018-08-23 06:00:40.804 505464 INFO cinder.manager [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Initiating service 4 cleanup >2018-08-23 06:00:40.809 505464 INFO cinder.manager [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Service 4 cleanup completed. >2018-08-23 06:00:41.005 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] WWPN on node 155: [u'500507680C1648C2', u'500507680C3648C2', u'500507680C3748C2', u'500507680C3848C2', u'500507680C1348C2', u'500507680C1848C2', u'500507680C3348C2', u'500507680C3548C2', u'500507680C1448C2', u'500507680C1548C2', u'500507680C1148C2', u'500507680C1748C2', u'500507680C1248C2', u'500507680C3248C2', u'500507680C3448C2', u'500507680C3148C2']. >2018-08-23 06:00:41.085 505464 INFO cinder.volume.manager [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 06:00:41.478 505464 INFO cinder.volume.manager [req-e3089168-728a-40e1-b292-547ee8b92782 - - - - -] Driver post RPC initialization completed successfully. >2018-08-23 06:00:41.913 505523 WARNING oslo_config.cfg [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:41.913 505523 WARNING oslo_config.cfg [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:41.914 505523 WARNING oslo_config.cfg [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 06:00:42.049 505523 INFO cinder.volume.manager [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Driver initialization completed successfully. >2018-08-23 06:00:42.056 505523 INFO cinder.manager [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Initiating service 5 cleanup >2018-08-23 06:00:42.060 505523 INFO cinder.manager [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Service 5 cleanup completed. >2018-08-23 06:00:42.337 505523 INFO cinder.volume.manager [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 06:00:42.551 505523 INFO cinder.volume.manager [req-7e54dc07-850f-4675-89e4-3d5d3a188fc3 - - - - -] Driver post RPC initialization completed successfully. >2018-08-23 06:03:32.200 505523 INFO cinder.volume.flows.manager.create_volume [req-923ce710-327d-4908-b193-d1d22fcb2ecd e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume 157398ef-7ee3-4092-8155-b9bda081955c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-157398ef-7ee3-4092-8155-b9bda081955c'} >2018-08-23 06:03:35.283 505523 INFO cinder.volume.flows.manager.create_volume [req-923ce710-327d-4908-b193-d1d22fcb2ecd e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume volume-157398ef-7ee3-4092-8155-b9bda081955c (157398ef-7ee3-4092-8155-b9bda081955c): created successfully >2018-08-23 06:03:35.287 505523 INFO cinder.volume.manager [req-923ce710-327d-4908-b193-d1d22fcb2ecd e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Created volume successfully. >2018-08-23 06:04:37.009 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-8d2d38f3-9cfa-4122-a7ab-242c233630eb e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Tried to delete non-existent vdisk volume-0a05592a-b368-4c8c-9430-a1c8b1e20893. >2018-08-23 06:04:37.312 505523 INFO cinder.volume.manager [req-8d2d38f3-9cfa-4122-a7ab-242c233630eb e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deleted volume successfully. >2018-08-23 06:18:29.900 505523 INFO cinder.volume.flows.manager.create_volume [req-ba9d5df3-0981-45a3-abee-077cadbe2c6e 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Volume 2570638e-5dc2-4005-8514-e3c81f553d1d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2570638e-5dc2-4005-8514-e3c81f553d1d'} >2018-08-23 06:18:30.518 505523 INFO cinder.volume.flows.manager.create_volume [req-ba9d5df3-0981-45a3-abee-077cadbe2c6e 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Volume volume-2570638e-5dc2-4005-8514-e3c81f553d1d (2570638e-5dc2-4005-8514-e3c81f553d1d): created successfully >2018-08-23 06:18:30.523 505523 INFO cinder.volume.manager [req-ba9d5df3-0981-45a3-abee-077cadbe2c6e 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Created volume successfully. >2018-08-23 06:18:37.600 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:18:37.692 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:18:41.476 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-36223c3f-0a17-4e16-88bf-daf488467051 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] initialize_connection: Did not find a preferred node for volume volume-2570638e-5dc2-4005-8514-e3c81f553d1d. >2018-08-23 06:18:41.927 505523 INFO cinder.volume.manager [req-36223c3f-0a17-4e16-88bf-daf488467051 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Initialize volume connection completed successfully. >2018-08-23 06:19:03.002 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-36223c3f-0a17-4e16-88bf-daf488467051 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:19:03.477 505523 INFO cinder.volume.manager [req-36223c3f-0a17-4e16-88bf-daf488467051 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Terminate volume connection completed successfully. >2018-08-23 06:19:03.521 505523 INFO cinder.volume.manager [req-36223c3f-0a17-4e16-88bf-daf488467051 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Remove volume export completed successfully. >2018-08-23 06:19:04.782 505523 INFO cinder.volume.flows.manager.create_volume [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Volume be509e03-2cf1-4d36-ba67-643f6a89e248: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-be509e03-2cf1-4d36-ba67-643f6a89e248'} >2018-08-23 06:19:05.711 505523 INFO cinder.volume.flows.manager.create_volume [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Volume volume-be509e03-2cf1-4d36-ba67-643f6a89e248 (be509e03-2cf1-4d36-ba67-643f6a89e248): created successfully >2018-08-23 06:19:05.714 505523 INFO cinder.volume.manager [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Created volume successfully. >2018-08-23 06:19:15.647 505523 INFO cinder.volume.manager [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Initialize volume connection completed successfully. >2018-08-23 06:19:22.678 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:19:22.955 505523 INFO cinder.volume.manager [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Terminate volume connection completed successfully. >2018-08-23 06:19:23.004 505523 INFO cinder.volume.manager [req-f6631af8-dd47-44f5-a2ad-4c63bda250f5 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Remove volume export completed successfully. >2018-08-23 06:19:24.435 505523 INFO cinder.volume.manager [req-69844db1-0673-436d-a7cd-0fe7d1c4e717 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Deleted volume successfully. >2018-08-23 06:19:25.056 505464 INFO cinder.volume.flows.manager.create_volume [req-3d6ed5bd-aa4e-442a-a697-37735e32ddec 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Volume 4477374f-3bbc-4f69-9b52-821485e3a1e4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4477374f-3bbc-4f69-9b52-821485e3a1e4'} >2018-08-23 06:19:26.008 505464 INFO cinder.volume.flows.manager.create_volume [req-3d6ed5bd-aa4e-442a-a697-37735e32ddec 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Volume volume-4477374f-3bbc-4f69-9b52-821485e3a1e4 (4477374f-3bbc-4f69-9b52-821485e3a1e4): created successfully >2018-08-23 06:19:26.014 505464 INFO cinder.volume.manager [req-3d6ed5bd-aa4e-442a-a697-37735e32ddec 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Created volume successfully. >2018-08-23 06:19:34.553 505464 INFO cinder.volume.manager [req-8933bafb-704b-4f01-a1f4-40e42d281570 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Initialize volume connection completed successfully. >2018-08-23 06:19:55.496 505464 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-8933bafb-704b-4f01-a1f4-40e42d281570 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:19:55.500 505464 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:19:55.562 505464 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:19:55.938 505464 INFO cinder.volume.manager [req-8933bafb-704b-4f01-a1f4-40e42d281570 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Terminate volume connection completed successfully. >2018-08-23 06:19:55.995 505464 INFO cinder.volume.manager [req-8933bafb-704b-4f01-a1f4-40e42d281570 036cdd22f06d422ba0ec80bdd059ea87 d8a6ef6fdf5943cb8bba8232e2635d8b - default default] Remove volume export completed successfully. >2018-08-23 06:19:58.797 505523 INFO cinder.volume.manager [req-458852c8-4796-438a-a937-e1411448ec29 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Deleted volume successfully. >2018-08-23 06:19:58.988 505464 INFO cinder.volume.manager [req-c7e99282-8629-4d0c-b6cb-b440e0106622 337f02b354f3411faadebd063c64b3e2 d29c9ce4a15d4477802e374768942175 - default default] Deleted volume successfully. >2018-08-23 06:20:04.660 505523 INFO cinder.volume.flows.manager.create_volume [req-2dd6567a-f864-4980-9e4f-38472d38b31d 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume 4f5a15e7-6182-463d-8537-1a4671e4569b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4f5a15e7-6182-463d-8537-1a4671e4569b'} >2018-08-23 06:20:05.203 505523 INFO cinder.volume.flows.manager.create_volume [req-2dd6567a-f864-4980-9e4f-38472d38b31d 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume volume-4f5a15e7-6182-463d-8537-1a4671e4569b (4f5a15e7-6182-463d-8537-1a4671e4569b): created successfully >2018-08-23 06:20:05.207 505523 INFO cinder.volume.manager [req-2dd6567a-f864-4980-9e4f-38472d38b31d 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Created volume successfully. >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 06:23:54.436 505523 ERROR oslo_service.periodic_task >2018-08-23 06:23:55.485 505464 ERROR cinder.ssh_utils [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:23:55.487 505464 ERROR cinder.ssh_utils [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:23:55.487 505464 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:23:55.488 505464 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:23:55.488 505464 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task command) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:23:55.488 505464 ERROR oslo_service.periodic_task >2018-08-23 06:24:55.485 505464 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:24:55.579 505464 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:25:07.587 505523 ERROR cinder.ssh_utils [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 06:25:07.589 505523 ERROR cinder.ssh_utils [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-23 06:25:07.590 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 06:25:07.590 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 06:25:07.591 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task command) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-23 06:25:07.591 505523 ERROR oslo_service.periodic_task >2018-08-23 06:25:09.507 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:25:09.575 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:25:10.974 505523 INFO cinder.volume.manager [req-b5ac38bc-dcfe-49f4-aeed-0f5c79c0b367 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Deleted volume successfully. >2018-08-23 06:25:11.242 505523 WARNING oslo_config.cfg [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 06:25:12.429 505523 INFO cinder.volume.flows.manager.create_volume [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume aa5f5f12-a737-49fa-a641-3862b3ef6d6b: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-aa5f5f12-a737-49fa-a641-3862b3ef6d6b', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f2695225210>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 06:25:13.863 505523 INFO cinder.image.image_utils [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Image download 12.00 MB at 8.38 MB/s >2018-08-23 06:25:15.803 505523 INFO oslo.privsep.daemon [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpwxiKeu/privsep.sock'] >2018-08-23 06:25:16.593 505523 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-23 06:25:16.708 505523 INFO oslo.privsep.daemon [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Spawned new privsep daemon via rootwrap >2018-08-23 06:25:16.638 655418 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-23 06:25:16.641 655418 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-23 06:25:16.644 655418 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-23 06:25:16.644 655418 INFO oslo.privsep.daemon [-] privsep daemon running as pid 655418 >2018-08-23 06:25:25.096 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] initialize_connection: Did not find a preferred node for volume volume-aa5f5f12-a737-49fa-a641-3862b3ef6d6b. >2018-08-23 06:25:25.742 505523 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 06:25:31.049 505523 INFO cinder.image.image_utils [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Converted 39.00 MB image at 36.52 MB/s >2018-08-23 06:25:32.792 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:25:33.218 505523 INFO cinder.volume.flows.manager.create_volume [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume volume-aa5f5f12-a737-49fa-a641-3862b3ef6d6b (aa5f5f12-a737-49fa-a641-3862b3ef6d6b): created successfully >2018-08-23 06:25:33.222 505523 INFO cinder.volume.manager [req-bda48d2a-f66f-4bcc-97de-d36f99147612 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Created volume successfully. >2018-08-23 06:25:37.592 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:25:37.652 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:25:41.496 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c126af14-1cba-46cc-abfb-7d009578a43b 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] initialize_connection: Did not find a preferred node for volume volume-aa5f5f12-a737-49fa-a641-3862b3ef6d6b. >2018-08-23 06:25:41.744 505523 INFO cinder.volume.manager [req-c126af14-1cba-46cc-abfb-7d009578a43b 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Initialize volume connection completed successfully. >2018-08-23 06:26:05.027 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c126af14-1cba-46cc-abfb-7d009578a43b 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:26:05.525 505523 INFO cinder.volume.manager [req-c126af14-1cba-46cc-abfb-7d009578a43b 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Terminate volume connection completed successfully. >2018-08-23 06:26:05.575 505523 INFO cinder.volume.manager [req-c126af14-1cba-46cc-abfb-7d009578a43b 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Remove volume export completed successfully. >2018-08-23 06:26:06.862 505523 INFO cinder.volume.flows.manager.create_volume [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume 54bc20af-eea7-4c76-af96-66f27662cd21: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-54bc20af-eea7-4c76-af96-66f27662cd21'} >2018-08-23 06:26:07.904 505523 INFO cinder.volume.flows.manager.create_volume [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume volume-54bc20af-eea7-4c76-af96-66f27662cd21 (54bc20af-eea7-4c76-af96-66f27662cd21): created successfully >2018-08-23 06:26:07.908 505523 INFO cinder.volume.manager [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Created volume successfully. >2018-08-23 06:26:22.497 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] initialize_connection: Did not find a preferred node for volume volume-54bc20af-eea7-4c76-af96-66f27662cd21. >2018-08-23 06:26:22.925 505523 INFO cinder.volume.manager [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Initialize volume connection completed successfully. >2018-08-23 06:26:32.859 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:26:33.322 505523 INFO cinder.volume.manager [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Terminate volume connection completed successfully. >2018-08-23 06:26:33.380 505523 INFO cinder.volume.manager [req-a7581304-672f-4dd8-b411-0c8b7320ec5e 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Remove volume export completed successfully. >2018-08-23 06:26:34.491 505523 INFO cinder.volume.flows.manager.create_volume [req-7be86ab8-9b04-476d-a90a-84144a3dbfb9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume 4baf0129-0e37-45bd-906f-cd241dbf31ad: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4baf0129-0e37-45bd-906f-cd241dbf31ad'} >2018-08-23 06:26:35.770 505523 INFO cinder.volume.flows.manager.create_volume [req-7be86ab8-9b04-476d-a90a-84144a3dbfb9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume volume-4baf0129-0e37-45bd-906f-cd241dbf31ad (4baf0129-0e37-45bd-906f-cd241dbf31ad): created successfully >2018-08-23 06:26:35.775 505523 INFO cinder.volume.manager [req-7be86ab8-9b04-476d-a90a-84144a3dbfb9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Created volume successfully. >2018-08-23 06:26:36.424 505523 INFO cinder.volume.manager [req-b7a87455-ff1d-40b6-942e-9818a8d0430c 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Deleted volume successfully. >2018-08-23 06:26:49.928 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b9dcc395-0d94-48b4-a8a8-9e3bb36d87d6 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] initialize_connection: Did not find a preferred node for volume volume-4baf0129-0e37-45bd-906f-cd241dbf31ad. >2018-08-23 06:26:50.350 505523 INFO cinder.volume.manager [req-b9dcc395-0d94-48b4-a8a8-9e3bb36d87d6 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Initialize volume connection completed successfully. >2018-08-23 06:27:13.272 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b9dcc395-0d94-48b4-a8a8-9e3bb36d87d6 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:27:13.766 505523 INFO cinder.volume.manager [req-b9dcc395-0d94-48b4-a8a8-9e3bb36d87d6 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Terminate volume connection completed successfully. >2018-08-23 06:27:13.817 505523 INFO cinder.volume.manager [req-b9dcc395-0d94-48b4-a8a8-9e3bb36d87d6 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Remove volume export completed successfully. >2018-08-23 06:27:14.657 505464 INFO cinder.volume.flows.manager.create_volume [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume cf6b087b-77c8-49d7-9796-28a6330a82ef: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cf6b087b-77c8-49d7-9796-28a6330a82ef'} >2018-08-23 06:27:15.654 505464 INFO cinder.volume.flows.manager.create_volume [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Volume volume-cf6b087b-77c8-49d7-9796-28a6330a82ef (cf6b087b-77c8-49d7-9796-28a6330a82ef): created successfully >2018-08-23 06:27:15.658 505464 INFO cinder.volume.manager [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Created volume successfully. >2018-08-23 06:27:31.316 505464 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] initialize_connection: Did not find a preferred node for volume volume-cf6b087b-77c8-49d7-9796-28a6330a82ef. >2018-08-23 06:27:31.793 505464 INFO cinder.volume.manager [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Initialize volume connection completed successfully. >2018-08-23 06:27:41.064 505464 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:27:41.441 505464 INFO cinder.volume.manager [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Terminate volume connection completed successfully. >2018-08-23 06:27:41.487 505464 INFO cinder.volume.manager [req-c42f842e-f546-4ff7-afa5-5ab4dff579e9 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Remove volume export completed successfully. >2018-08-23 06:27:44.732 505464 INFO cinder.volume.manager [req-c3d420bb-3614-48a5-9ea6-80c43be93b12 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Deleted volume successfully. >2018-08-23 06:27:44.752 505523 INFO cinder.volume.manager [req-e33cce50-191f-43cb-9dea-392ed8c10a93 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Deleted volume successfully. >2018-08-23 06:27:44.987 505523 INFO cinder.volume.manager [req-a65a4dc0-3b2a-4554-82ef-94a4e96ed57c 58d054cf42fc4f25b62fd3adb1174758 f5e75c26fc7846969651d0657cf9708e - default default] Deleted volume successfully. >2018-08-23 06:27:50.517 505523 INFO cinder.volume.flows.manager.create_volume [req-02024792-6ef6-465c-9c0a-33c91ffca27c 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Volume 6d64424b-6ecb-47e2-9c6b-360b3f43f101: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6d64424b-6ecb-47e2-9c6b-360b3f43f101'} >2018-08-23 06:27:51.132 505523 INFO cinder.volume.flows.manager.create_volume [req-02024792-6ef6-465c-9c0a-33c91ffca27c 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Volume volume-6d64424b-6ecb-47e2-9c6b-360b3f43f101 (6d64424b-6ecb-47e2-9c6b-360b3f43f101): created successfully >2018-08-23 06:27:51.135 505523 INFO cinder.volume.manager [req-02024792-6ef6-465c-9c0a-33c91ffca27c 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Created volume successfully. >2018-08-23 06:28:00.677 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-014abb99-791e-4f1b-b6b4-bd641c5e7ff9 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] initialize_connection: Did not find a preferred node for volume volume-6d64424b-6ecb-47e2-9c6b-360b3f43f101. >2018-08-23 06:28:00.922 505523 INFO cinder.volume.manager [req-014abb99-791e-4f1b-b6b4-bd641c5e7ff9 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Initialize volume connection completed successfully. >2018-08-23 06:28:21.972 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-014abb99-791e-4f1b-b6b4-bd641c5e7ff9 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 06:28:22.281 505523 INFO cinder.volume.manager [req-014abb99-791e-4f1b-b6b4-bd641c5e7ff9 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Terminate volume connection completed successfully. >2018-08-23 06:28:22.325 505523 INFO cinder.volume.manager [req-014abb99-791e-4f1b-b6b4-bd641c5e7ff9 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Remove volume export completed successfully. >2018-08-23 06:28:24.816 505523 INFO cinder.volume.manager [req-b6dc023a-331d-4cf7-b587-5b0ea17fe9a0 78c7887f9a9142aa8aa463599080444f 677d4498743e4be5afd625260437de02 - default default] Deleted volume successfully. >2018-08-23 06:43:30.603 505523 INFO cinder.volume.manager [req-c0909270-aac0-4cf7-9dc9-e4fcb553545a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-1928168850: creating >2018-08-23 06:43:30.630 505523 INFO cinder.volume.manager [req-c0909270-aac0-4cf7-9dc9-e4fcb553545a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-1928168850: created successfully >2018-08-23 06:43:30.650 505523 INFO cinder.volume.manager [req-c0909270-aac0-4cf7-9dc9-e4fcb553545a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Create group completed successfully. >2018-08-23 06:43:31.776 505523 INFO cinder.volume.flows.manager.create_volume [req-275561d4-1f99-4c8d-bed3-608a97fd3021 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume b52f7ec0-fc48-4528-92c7-7063008a1525: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b52f7ec0-fc48-4528-92c7-7063008a1525'} >2018-08-23 06:43:32.335 505523 INFO cinder.volume.flows.manager.create_volume [req-275561d4-1f99-4c8d-bed3-608a97fd3021 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume volume-b52f7ec0-fc48-4528-92c7-7063008a1525 (b52f7ec0-fc48-4528-92c7-7063008a1525): created successfully >2018-08-23 06:43:32.338 505523 INFO cinder.volume.manager [req-275561d4-1f99-4c8d-bed3-608a97fd3021 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Created volume successfully. >2018-08-23 06:43:34.645 505523 INFO cinder.volume.manager [req-13b4390d-6415-4e8a-81b3-49c638ef6a1e eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] GroupSnapshot 3a8fd07f-0a2e-4925-b32a-36fec3ae97c3: creating. >2018-08-23 06:43:37.655 505523 INFO cinder.volume.manager [req-13b4390d-6415-4e8a-81b3-49c638ef6a1e eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] group_snapshot 3a8fd07f-0a2e-4925-b32a-36fec3ae97c3: created successfully >2018-08-23 06:43:39.750 505523 INFO cinder.volume.manager [req-c8a9ff40-6e8e-4367-be7c-d06154c64407 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] group_snapshot 3a8fd07f-0a2e-4925-b32a-36fec3ae97c3: deleting >2018-08-23 06:43:41.543 505523 WARNING cinder.quota [req-c8a9ff40-6e8e-4367-be7c-d06154c64407 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-796104343 is set by the default quota flag: quota_snapshots_tempest-volume-type-796104343, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:43:41.574 505523 INFO cinder.volume.manager [req-c8a9ff40-6e8e-4367-be7c-d06154c64407 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] group_snapshot 3a8fd07f-0a2e-4925-b32a-36fec3ae97c3: deleted successfully >2018-08-23 06:43:43.091 505523 WARNING cinder.quota [req-eca4fd34-dab5-4f68-a6bd-fcf088f94d3a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-796104343 is set by the default quota flag: quota_volumes_tempest-volume-type-796104343, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:43:43.092 505523 WARNING cinder.quota [req-eca4fd34-dab5-4f68-a6bd-fcf088f94d3a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-796104343 is set by the default quota flag: quota_gigabytes_tempest-volume-type-796104343, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:43:43.412 505523 INFO cinder.volume.manager [req-eca4fd34-dab5-4f68-a6bd-fcf088f94d3a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Delete group completed successfully. >2018-08-23 06:43:43.571 505523 INFO cinder.volume.manager [req-ac558a6d-c450-44ae-a4b6-b87b45e4e955 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-84732809: creating >2018-08-23 06:43:43.592 505523 INFO cinder.volume.manager [req-ac558a6d-c450-44ae-a4b6-b87b45e4e955 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-84732809: created successfully >2018-08-23 06:43:43.615 505523 INFO cinder.volume.manager [req-ac558a6d-c450-44ae-a4b6-b87b45e4e955 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Create group completed successfully. >2018-08-23 06:43:43.990 505523 INFO cinder.volume.flows.manager.create_volume [req-58f46816-57d3-4825-b557-9d4e92a997af eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume ba84eb10-cf1f-4a56-b40d-5b92648f0dd2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ba84eb10-cf1f-4a56-b40d-5b92648f0dd2'} >2018-08-23 06:43:44.616 505523 INFO cinder.volume.flows.manager.create_volume [req-58f46816-57d3-4825-b557-9d4e92a997af eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume volume-ba84eb10-cf1f-4a56-b40d-5b92648f0dd2 (ba84eb10-cf1f-4a56-b40d-5b92648f0dd2): created successfully >2018-08-23 06:43:44.620 505523 INFO cinder.volume.manager [req-58f46816-57d3-4825-b557-9d4e92a997af eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Created volume successfully. >2018-08-23 06:43:46.915 505523 WARNING cinder.quota [req-deed0588-64db-4524-a66e-8446e317ee6a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1922185820 is set by the default quota flag: quota_volumes_tempest-volume-type-1922185820, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:43:46.916 505523 WARNING cinder.quota [req-deed0588-64db-4524-a66e-8446e317ee6a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1922185820 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1922185820, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:43:47.191 505523 INFO cinder.volume.manager [req-deed0588-64db-4524-a66e-8446e317ee6a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Delete group completed successfully. >2018-08-23 06:43:47.505 505523 INFO cinder.volume.manager [req-f8e17e8f-10e0-400c-9f2e-a33344229a13 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-1199014968: creating >2018-08-23 06:43:47.527 505523 INFO cinder.volume.manager [req-f8e17e8f-10e0-400c-9f2e-a33344229a13 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-1199014968: created successfully >2018-08-23 06:43:47.548 505523 INFO cinder.volume.manager [req-f8e17e8f-10e0-400c-9f2e-a33344229a13 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Create group completed successfully. >2018-08-23 06:43:47.973 505523 INFO cinder.volume.flows.manager.create_volume [req-8900cb48-7178-4d22-b4be-46a1d5d625b9 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume b9b55b9c-9dba-4ed5-a1e7-64ffaf4c45dd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b9b55b9c-9dba-4ed5-a1e7-64ffaf4c45dd'} >2018-08-23 06:43:48.519 505523 INFO cinder.volume.flows.manager.create_volume [req-8900cb48-7178-4d22-b4be-46a1d5d625b9 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume volume-b9b55b9c-9dba-4ed5-a1e7-64ffaf4c45dd (b9b55b9c-9dba-4ed5-a1e7-64ffaf4c45dd): created successfully >2018-08-23 06:43:48.523 505523 INFO cinder.volume.manager [req-8900cb48-7178-4d22-b4be-46a1d5d625b9 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Created volume successfully. >2018-08-23 06:43:49.395 505523 INFO cinder.volume.manager [req-08b88cec-0d91-42ee-87bc-dfc89e4e48c9 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] GroupSnapshot 2bb091e6-fd10-4361-bf82-4514703e2455: creating. >2018-08-23 06:43:52.820 505523 INFO cinder.volume.manager [req-08b88cec-0d91-42ee-87bc-dfc89e4e48c9 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] group_snapshot 2bb091e6-fd10-4361-bf82-4514703e2455: created successfully >2018-08-23 06:43:56.498 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-be61e642-05cf-4068-9e10-8b66308b2c35 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Updating status for CG: afdc3461-9407-4fe1-ae53-24adddc953ce. >2018-08-23 06:43:56.603 505523 INFO cinder.volume.manager [req-be61e642-05cf-4068-9e10-8b66308b2c35 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Create group from source-snapshot-2bb091e6-fd10-4361-bf82-4514703e2455 completed successfully. >2018-08-23 06:44:01.032 505523 WARNING cinder.quota [req-95b1045c-3d57-4c88-9eb8-5000048ec3e7 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-2107632321 is set by the default quota flag: quota_volumes_tempest-volume-type-2107632321, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:44:01.033 505523 WARNING cinder.quota [req-95b1045c-3d57-4c88-9eb8-5000048ec3e7 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-2107632321 is set by the default quota flag: quota_gigabytes_tempest-volume-type-2107632321, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:44:01.317 505523 INFO cinder.volume.manager [req-95b1045c-3d57-4c88-9eb8-5000048ec3e7 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Delete group completed successfully. >2018-08-23 06:44:01.633 505523 INFO cinder.volume.manager [req-cf9c7e99-47b0-4b33-af57-beaedc82cc3a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] group_snapshot 2bb091e6-fd10-4361-bf82-4514703e2455: deleting >2018-08-23 06:44:03.458 505523 WARNING cinder.quota [req-cf9c7e99-47b0-4b33-af57-beaedc82cc3a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-2107632321 is set by the default quota flag: quota_snapshots_tempest-volume-type-2107632321, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:44:03.493 505523 INFO cinder.volume.manager [req-cf9c7e99-47b0-4b33-af57-beaedc82cc3a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] group_snapshot 2bb091e6-fd10-4361-bf82-4514703e2455: deleted successfully >2018-08-23 06:44:05.226 505523 INFO cinder.volume.manager [req-52a8b282-309a-4bdc-827d-fba9b024ebe3 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Delete group completed successfully. >2018-08-23 06:44:05.545 505523 INFO cinder.volume.manager [req-41d9c15c-c97e-4566-9050-19d0d054748a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-132477923: creating >2018-08-23 06:44:05.576 505523 INFO cinder.volume.manager [req-41d9c15c-c97e-4566-9050-19d0d054748a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Group tempest-CG-132477923: created successfully >2018-08-23 06:44:05.602 505523 INFO cinder.volume.manager [req-41d9c15c-c97e-4566-9050-19d0d054748a eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Create group completed successfully. >2018-08-23 06:44:05.983 505523 INFO cinder.volume.flows.manager.create_volume [req-c9412694-21d8-4f9d-8d60-e5fe696ec823 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume 2be3b6da-ba51-4b7a-b80a-036ef608872d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2be3b6da-ba51-4b7a-b80a-036ef608872d'} >2018-08-23 06:44:06.532 505523 INFO cinder.volume.flows.manager.create_volume [req-c9412694-21d8-4f9d-8d60-e5fe696ec823 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Volume volume-2be3b6da-ba51-4b7a-b80a-036ef608872d (2be3b6da-ba51-4b7a-b80a-036ef608872d): created successfully >2018-08-23 06:44:06.536 505523 INFO cinder.volume.manager [req-c9412694-21d8-4f9d-8d60-e5fe696ec823 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Created volume successfully. >2018-08-23 06:44:10.399 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-d1efd3ec-b481-49e6-9016-4a2237e31f85 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Updating status for CG: 098b1288-287e-4b10-a195-f22bef3ae236. >2018-08-23 06:44:10.493 505523 INFO cinder.volume.manager [req-d1efd3ec-b481-49e6-9016-4a2237e31f85 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Create group from source-group-36cbcd1a-8b8e-45b3-b1fc-dcce54ddf464 completed successfully. >2018-08-23 06:44:15.062 505523 WARNING cinder.quota [req-c9973c53-17ef-4c8d-8b47-294f1a154c93 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1409470063 is set by the default quota flag: quota_volumes_tempest-volume-type-1409470063, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:44:15.063 505523 WARNING cinder.quota [req-c9973c53-17ef-4c8d-8b47-294f1a154c93 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1409470063 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1409470063, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:44:15.358 505523 INFO cinder.volume.manager [req-c9973c53-17ef-4c8d-8b47-294f1a154c93 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Delete group completed successfully. >2018-08-23 06:44:17.438 505523 INFO cinder.volume.manager [req-7fb5e5be-89a3-44be-bb36-ffeea3f27df8 eb519cdf3d824647bc73250d6d8c61aa 80656b6a9d474040bd1109c71cae602f - default default] Delete group completed successfully. >2018-08-23 06:44:59.575 505523 INFO cinder.volume.flows.manager.create_volume [req-51dfe4b5-7d29-4832-9006-eec66964408a 86c475e8c4c4420e933d1960690032bd 97e63911162f4e3b852d3994c0d97cf1 - default default] Volume 4d9dfa58-42aa-4b96-a13c-abad86d6a82b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4d9dfa58-42aa-4b96-a13c-abad86d6a82b'} >2018-08-23 06:45:00.124 505523 INFO cinder.volume.flows.manager.create_volume [req-51dfe4b5-7d29-4832-9006-eec66964408a 86c475e8c4c4420e933d1960690032bd 97e63911162f4e3b852d3994c0d97cf1 - default default] Volume volume-4d9dfa58-42aa-4b96-a13c-abad86d6a82b (4d9dfa58-42aa-4b96-a13c-abad86d6a82b): created successfully >2018-08-23 06:45:00.127 505523 INFO cinder.volume.manager [req-51dfe4b5-7d29-4832-9006-eec66964408a 86c475e8c4c4420e933d1960690032bd 97e63911162f4e3b852d3994c0d97cf1 - default default] Created volume successfully. >2018-08-23 06:45:02.532 505523 INFO cinder.volume.manager [req-65264048-fec1-4cdb-b60d-ea7f24b3cf91 86c475e8c4c4420e933d1960690032bd 97e63911162f4e3b852d3994c0d97cf1 - default default] Deleted volume successfully. >2018-08-23 06:45:12.481 505523 INFO cinder.volume.flows.manager.create_volume [req-64069414-8ba0-4fc8-8294-2987db00689b 164977ad3e604e8f828c374ef6c39d4a 41607b662d824a028345533c4f310435 - default default] Volume ae94893e-bcb9-484d-8974-fa797ba604df: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ae94893e-bcb9-484d-8974-fa797ba604df'} >2018-08-23 06:45:13.108 505523 INFO cinder.volume.flows.manager.create_volume [req-64069414-8ba0-4fc8-8294-2987db00689b 164977ad3e604e8f828c374ef6c39d4a 41607b662d824a028345533c4f310435 - default default] Volume volume-ae94893e-bcb9-484d-8974-fa797ba604df (ae94893e-bcb9-484d-8974-fa797ba604df): created successfully >2018-08-23 06:45:13.112 505523 INFO cinder.volume.manager [req-64069414-8ba0-4fc8-8294-2987db00689b 164977ad3e604e8f828c374ef6c39d4a 41607b662d824a028345533c4f310435 - default default] Created volume successfully. >2018-08-23 06:45:15.843 505523 WARNING cinder.quota [req-fca3daef-f33a-4699-9497-617a26aef93e 164977ad3e604e8f828c374ef6c39d4a 41607b662d824a028345533c4f310435 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1478165919 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1478165919, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:45:15.845 505523 WARNING cinder.quota [req-fca3daef-f33a-4699-9497-617a26aef93e 164977ad3e604e8f828c374ef6c39d4a 41607b662d824a028345533c4f310435 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1478165919 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1478165919, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:45:16.093 505523 INFO cinder.volume.manager [req-fca3daef-f33a-4699-9497-617a26aef93e 164977ad3e604e8f828c374ef6c39d4a 41607b662d824a028345533c4f310435 - default default] Deleted volume successfully. >2018-08-23 06:45:25.640 505523 INFO cinder.volume.flows.manager.create_volume [req-565c9ead-fc7d-4a83-bfaf-71296542140f 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] Volume 4f7fe24b-3a77-44a7-8009-90bb59599c59: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4f7fe24b-3a77-44a7-8009-90bb59599c59'} >2018-08-23 06:45:26.185 505523 INFO cinder.volume.flows.manager.create_volume [req-565c9ead-fc7d-4a83-bfaf-71296542140f 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] Volume volume-4f7fe24b-3a77-44a7-8009-90bb59599c59 (4f7fe24b-3a77-44a7-8009-90bb59599c59): created successfully >2018-08-23 06:45:26.189 505523 INFO cinder.volume.manager [req-565c9ead-fc7d-4a83-bfaf-71296542140f 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] Created volume successfully. >2018-08-23 06:45:27.606 505523 INFO cinder.volume.manager [req-223b9d0b-32bd-409c-be53-50ca5b064cf9 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] Retype volume completed successfully. >2018-08-23 06:45:31.364 505523 WARNING cinder.quota [req-64fdaa00-962b-41a7-995b-e08e8c73207c 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-720077256 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-720077256, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:45:31.365 505523 WARNING cinder.quota [req-64fdaa00-962b-41a7-995b-e08e8c73207c 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-720077256 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-720077256, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 06:45:31.617 505523 INFO cinder.volume.manager [req-64fdaa00-962b-41a7-995b-e08e8c73207c 925fec5edb594132aef26a77879d958c e0b3498df1fb4822bde088b52dc590ae - default default] Deleted volume successfully. >2018-08-23 06:51:10.865 505523 INFO cinder.volume.flows.manager.create_volume [req-e0a1d187-8ccc-45ac-9eab-ff8e76cf4249 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume 27c653af-b7c9-463b-8124-c7cde92a0564: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-27c653af-b7c9-463b-8124-c7cde92a0564'} >2018-08-23 06:51:11.390 505523 INFO cinder.volume.flows.manager.create_volume [req-e0a1d187-8ccc-45ac-9eab-ff8e76cf4249 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume volume-27c653af-b7c9-463b-8124-c7cde92a0564 (27c653af-b7c9-463b-8124-c7cde92a0564): created successfully >2018-08-23 06:51:11.394 505523 INFO cinder.volume.manager [req-e0a1d187-8ccc-45ac-9eab-ff8e76cf4249 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Created volume successfully. >2018-08-23 06:51:14.265 505523 INFO cinder.volume.manager [req-dafe58cc-f2e9-4381-8168-8cfe43cd3603 52a5389d811e4eefaa5f8bbe23df4241 a39e7289dda74751bea943e39082a38c - default default] Deleted volume successfully. >2018-08-23 06:51:15.462 505523 INFO cinder.volume.flows.manager.create_volume [req-2a94d26d-7a60-4e47-a017-67c5138ae828 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume 9ccc52cc-2aab-4a16-9cb2-53cf4b929d7b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9ccc52cc-2aab-4a16-9cb2-53cf4b929d7b'} >2018-08-23 06:51:15.980 505523 INFO cinder.volume.flows.manager.create_volume [req-2a94d26d-7a60-4e47-a017-67c5138ae828 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume volume-9ccc52cc-2aab-4a16-9cb2-53cf4b929d7b (9ccc52cc-2aab-4a16-9cb2-53cf4b929d7b): created successfully >2018-08-23 06:51:15.984 505523 INFO cinder.volume.manager [req-2a94d26d-7a60-4e47-a017-67c5138ae828 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Created volume successfully. >2018-08-23 06:51:18.410 505523 INFO cinder.volume.manager [req-bc548216-5fc8-4f6c-a593-ae4b7616de99 52a5389d811e4eefaa5f8bbe23df4241 a39e7289dda74751bea943e39082a38c - default default] Deleted volume successfully. >2018-08-23 06:51:19.020 505523 INFO cinder.volume.flows.manager.create_volume [req-6242c64c-0310-4ff8-b8b0-004c64f0d860 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume 78c0d2f9-02af-429e-8d66-92ba3f6b02ff: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-78c0d2f9-02af-429e-8d66-92ba3f6b02ff'} >2018-08-23 06:51:19.546 505523 INFO cinder.volume.flows.manager.create_volume [req-6242c64c-0310-4ff8-b8b0-004c64f0d860 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume volume-78c0d2f9-02af-429e-8d66-92ba3f6b02ff (78c0d2f9-02af-429e-8d66-92ba3f6b02ff): created successfully >2018-08-23 06:51:19.550 505523 INFO cinder.volume.manager [req-6242c64c-0310-4ff8-b8b0-004c64f0d860 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Created volume successfully. >2018-08-23 06:51:22.197 505523 INFO cinder.volume.manager [req-47326f3a-fa85-4f08-97f2-88aaed29a95c 52a5389d811e4eefaa5f8bbe23df4241 a39e7289dda74751bea943e39082a38c - default default] Deleted volume successfully. >2018-08-23 06:51:22.729 505523 INFO cinder.volume.flows.manager.create_volume [req-8a3bdfdc-b143-4e6f-ad50-7980b0a65eef 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume d1df782e-f790-4dde-b848-99ee0c41e9c6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d1df782e-f790-4dde-b848-99ee0c41e9c6'} >2018-08-23 06:51:23.251 505523 INFO cinder.volume.flows.manager.create_volume [req-8a3bdfdc-b143-4e6f-ad50-7980b0a65eef 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume volume-d1df782e-f790-4dde-b848-99ee0c41e9c6 (d1df782e-f790-4dde-b848-99ee0c41e9c6): created successfully >2018-08-23 06:51:23.255 505523 INFO cinder.volume.manager [req-8a3bdfdc-b143-4e6f-ad50-7980b0a65eef 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Created volume successfully. >2018-08-23 06:51:25.444 505523 INFO cinder.volume.manager [req-ee4a1cb8-efdc-4289-b5f4-a47d66dd8e17 52a5389d811e4eefaa5f8bbe23df4241 a39e7289dda74751bea943e39082a38c - default default] Deleted volume successfully. >2018-08-23 06:51:25.801 505523 INFO cinder.volume.flows.manager.create_volume [req-492c5833-e553-4622-a085-20fd23b21ee6 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume ab810b5f-27d1-410a-9050-87a5d3a9abb1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ab810b5f-27d1-410a-9050-87a5d3a9abb1'} >2018-08-23 06:51:26.326 505523 INFO cinder.volume.flows.manager.create_volume [req-492c5833-e553-4622-a085-20fd23b21ee6 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Volume volume-ab810b5f-27d1-410a-9050-87a5d3a9abb1 (ab810b5f-27d1-410a-9050-87a5d3a9abb1): created successfully >2018-08-23 06:51:26.330 505523 INFO cinder.volume.manager [req-492c5833-e553-4622-a085-20fd23b21ee6 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Created volume successfully. >2018-08-23 06:51:29.254 505523 INFO cinder.volume.manager [req-3ef2820d-5991-4b76-8885-592e27302d4f 923621e3bd6641fe9ebbfcb0ee075b27 edb5ae4ccfd746f1850943f4769026f1 - default default] Deleted volume successfully. >2018-08-23 06:51:37.116 505523 INFO cinder.volume.flows.manager.create_volume [req-1188e7d6-3c5e-479b-bc6f-bf854ca55de9 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Volume 31562339-901a-4c50-a088-784c1c8b0958: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-31562339-901a-4c50-a088-784c1c8b0958'} >2018-08-23 06:51:37.650 505523 INFO cinder.volume.flows.manager.create_volume [req-1188e7d6-3c5e-479b-bc6f-bf854ca55de9 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Volume volume-31562339-901a-4c50-a088-784c1c8b0958 (31562339-901a-4c50-a088-784c1c8b0958): created successfully >2018-08-23 06:51:37.654 505523 INFO cinder.volume.manager [req-1188e7d6-3c5e-479b-bc6f-bf854ca55de9 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Created volume successfully. >2018-08-23 06:51:39.573 505523 INFO cinder.volume.flows.manager.create_volume [req-8fbbc669-4974-4033-9665-ded47551ef6f ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Volume e2f5490e-47fb-4d6d-baa0-894a22173b10: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e2f5490e-47fb-4d6d-baa0-894a22173b10'} >2018-08-23 06:51:40.094 505523 INFO cinder.volume.flows.manager.create_volume [req-8fbbc669-4974-4033-9665-ded47551ef6f ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Volume volume-e2f5490e-47fb-4d6d-baa0-894a22173b10 (e2f5490e-47fb-4d6d-baa0-894a22173b10): created successfully >2018-08-23 06:51:40.098 505523 INFO cinder.volume.manager [req-8fbbc669-4974-4033-9665-ded47551ef6f ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Created volume successfully. >2018-08-23 06:51:41.231 505464 INFO cinder.volume.flows.manager.create_volume [req-f95aaf16-bdc1-4070-8bb7-0992bfd61db4 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Volume bc598132-3aa9-44cb-9ff0-4f7883e1b4d2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bc598132-3aa9-44cb-9ff0-4f7883e1b4d2'} >2018-08-23 06:51:41.844 505464 INFO cinder.volume.flows.manager.create_volume [req-f95aaf16-bdc1-4070-8bb7-0992bfd61db4 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Volume volume-bc598132-3aa9-44cb-9ff0-4f7883e1b4d2 (bc598132-3aa9-44cb-9ff0-4f7883e1b4d2): created successfully >2018-08-23 06:51:41.848 505464 INFO cinder.volume.manager [req-f95aaf16-bdc1-4070-8bb7-0992bfd61db4 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Created volume successfully. >2018-08-23 06:51:43.123 505523 INFO cinder.volume.flows.manager.create_volume [req-4b12b551-f926-4d02-8a72-fcef3d78e94c aeb45b8a6f9f4140b46475ec5612749f 6e671c8052b744918179ac16e7f21596 - default default] Volume 346b1202-1c8e-4734-b076-8a6aa70c68d2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-346b1202-1c8e-4734-b076-8a6aa70c68d2'} >2018-08-23 06:51:43.643 505523 INFO cinder.volume.flows.manager.create_volume [req-4b12b551-f926-4d02-8a72-fcef3d78e94c aeb45b8a6f9f4140b46475ec5612749f 6e671c8052b744918179ac16e7f21596 - default default] Volume volume-346b1202-1c8e-4734-b076-8a6aa70c68d2 (346b1202-1c8e-4734-b076-8a6aa70c68d2): created successfully >2018-08-23 06:51:43.647 505523 INFO cinder.volume.manager [req-4b12b551-f926-4d02-8a72-fcef3d78e94c aeb45b8a6f9f4140b46475ec5612749f 6e671c8052b744918179ac16e7f21596 - default default] Created volume successfully. >2018-08-23 06:51:45.888 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:51:45.963 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:51:47.138 505523 INFO cinder.volume.manager [req-2609ac28-c669-4deb-b101-b5317550dcc5 aeb45b8a6f9f4140b46475ec5612749f 6e671c8052b744918179ac16e7f21596 - default default] Deleted volume successfully. >2018-08-23 06:51:48.234 505523 INFO cinder.volume.manager [req-82bdf347-c813-467b-87ae-bd35f1111258 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Deleted volume successfully. >2018-08-23 06:51:48.272 505464 INFO cinder.volume.manager [req-1607abde-d1fb-42a2-8d54-e1a927281f4e ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Deleted volume successfully. >2018-08-23 06:51:48.474 505523 INFO cinder.volume.manager [req-a3a72c05-aca8-4671-bab2-bbb455925d50 ef9fac23302a4ae5844215637d295ab1 813423447f72454cb37ae986c03111db - default default] Deleted volume successfully. >2018-08-23 06:51:54.277 505523 INFO cinder.volume.flows.manager.create_volume [req-bdd2c5bc-a097-4b35-ab9b-9957468bc970 1c60f99d445b48b7b0548635b3eb0ad3 4bc1c13e1a7e4526869f66b57a657a89 - default default] Volume e466e024-a420-45f1-aa3a-f553d347f1a3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e466e024-a420-45f1-aa3a-f553d347f1a3'} >2018-08-23 06:51:54.824 505523 INFO cinder.volume.flows.manager.create_volume [req-bdd2c5bc-a097-4b35-ab9b-9957468bc970 1c60f99d445b48b7b0548635b3eb0ad3 4bc1c13e1a7e4526869f66b57a657a89 - default default] Volume volume-e466e024-a420-45f1-aa3a-f553d347f1a3 (e466e024-a420-45f1-aa3a-f553d347f1a3): created successfully >2018-08-23 06:51:54.828 505523 INFO cinder.volume.manager [req-bdd2c5bc-a097-4b35-ab9b-9957468bc970 1c60f99d445b48b7b0548635b3eb0ad3 4bc1c13e1a7e4526869f66b57a657a89 - default default] Created volume successfully. >2018-08-23 06:51:58.798 505523 INFO cinder.volume.manager [req-9cd286d4-b1dd-4175-af29-caf3be880d01 1c60f99d445b48b7b0548635b3eb0ad3 4bc1c13e1a7e4526869f66b57a657a89 - default default] Deleted volume successfully. >2018-08-23 06:52:06.609 505523 INFO cinder.volume.flows.manager.create_volume [req-bd6120de-c0c9-4062-bd01-534db6022756 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] Volume f80c2ec1-36ed-4574-83bb-31f565487f63: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f80c2ec1-36ed-4574-83bb-31f565487f63'} >2018-08-23 06:52:07.146 505523 INFO cinder.volume.flows.manager.create_volume [req-bd6120de-c0c9-4062-bd01-534db6022756 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] Volume volume-f80c2ec1-36ed-4574-83bb-31f565487f63 (f80c2ec1-36ed-4574-83bb-31f565487f63): created successfully >2018-08-23 06:52:07.149 505523 INFO cinder.volume.manager [req-bd6120de-c0c9-4062-bd01-534db6022756 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] Created volume successfully. >2018-08-23 06:52:09.320 505523 INFO cinder.volume.manager [req-cc9fee0e-4d70-423e-894a-93bdc2f9fd5c c47ebeef1d5d4370bbc9e16f85b70847 aaf4eb1bce354ef985daefc69b72dc18 - default default] Transfer volume completed successfully. >2018-08-23 06:52:11.383 505523 INFO cinder.volume.manager [req-bef3b002-0c7e-486e-ae12-046236bf79a9 a3f4820d605045869f256ac3f205e3af c907f33acb2a4947999f90e3adcf2688 - default default] Deleted volume successfully. >2018-08-23 06:52:12.215 505523 INFO cinder.volume.flows.manager.create_volume [req-ad215b6f-9313-40b8-97bc-5be581afde89 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] Volume 54da064f-5803-45dc-85cc-df991042daf7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-54da064f-5803-45dc-85cc-df991042daf7'} >2018-08-23 06:52:12.800 505523 INFO cinder.volume.flows.manager.create_volume [req-ad215b6f-9313-40b8-97bc-5be581afde89 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] Volume volume-54da064f-5803-45dc-85cc-df991042daf7 (54da064f-5803-45dc-85cc-df991042daf7): created successfully >2018-08-23 06:52:12.804 505523 INFO cinder.volume.manager [req-ad215b6f-9313-40b8-97bc-5be581afde89 7100fe470ec94dd6857058e983d948d2 88e7fbc290f74e90a4db168283f86e44 - default default] Created volume successfully. >2018-08-23 06:52:15.940 505523 INFO cinder.volume.manager [req-f0e1e636-7ab9-4eff-ba68-89a9b0f11278 a3f4820d605045869f256ac3f205e3af c907f33acb2a4947999f90e3adcf2688 - default default] Deleted volume successfully. >2018-08-23 06:52:22.716 505523 INFO cinder.volume.flows.manager.create_volume [req-107d28db-f8ab-47b0-a922-8aa9e9b46a08 d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] Volume de8669e6-eac4-42ad-b5b6-e47d1abd65fa: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-de8669e6-eac4-42ad-b5b6-e47d1abd65fa'} >2018-08-23 06:52:23.253 505523 INFO cinder.volume.flows.manager.create_volume [req-107d28db-f8ab-47b0-a922-8aa9e9b46a08 d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] Volume volume-de8669e6-eac4-42ad-b5b6-e47d1abd65fa (de8669e6-eac4-42ad-b5b6-e47d1abd65fa): created successfully >2018-08-23 06:52:23.257 505523 INFO cinder.volume.manager [req-107d28db-f8ab-47b0-a922-8aa9e9b46a08 d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] Created volume successfully. >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 06:54:58.034 505523 ERROR oslo_service.periodic_task >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task return self.result[key] >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-23 06:54:58.033 505464 ERROR oslo_service.periodic_task >2018-08-23 06:54:58.037 505464 WARNING oslo.service.loopingcall [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 2.54 sec >2018-08-23 06:54:58.038 505523 WARNING oslo.service.loopingcall [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 20.46 sec >2018-08-23 06:54:58.044 505464 ERROR cinder.ssh_utils [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.044 505523 ERROR cinder.ssh_utils [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.045 505523 ERROR cinder.ssh_utils [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.045 505464 ERROR cinder.ssh_utils [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.045 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.045 505464 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.045 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.045 505464 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.046 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.046 505464 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task [req-3193a320-fcdc-43ee-9adb-775d408d4a7d - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task command) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.046 505523 ERROR oslo_service.periodic_task >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task [req-c5a5ea34-5f3e-4a35-93b1-7881670a3189 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task task(self, context) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task command) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task obj = self.get() >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-23 06:54:58.046 505464 ERROR oslo_service.periodic_task >2018-08-23 06:55:58.107 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:55:58.107 505464 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:55:58.811 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:55:58.811 505464 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 06:56:58.055 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 06:56:58.134 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 07:02:40.900 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c779dfa6-f5a1-40f5-887d-0d6d7bcfdc0b d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] initialize_connection: Did not find a preferred node for volume volume-de8669e6-eac4-42ad-b5b6-e47d1abd65fa. >2018-08-23 07:02:41.489 505523 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 07:02:59.245 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-c779dfa6-f5a1-40f5-887d-0d6d7bcfdc0b d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 07:02:59.596 505523 INFO cinder.volume.manager [req-c779dfa6-f5a1-40f5-887d-0d6d7bcfdc0b d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] Copy volume to image completed successfully. >2018-08-23 07:03:03.029 505523 INFO cinder.volume.manager [req-d4e2743b-5ce5-4b95-bc32-24326d547f2c d235f3ba38474394b8faef948f5a7821 2caa13fa9b0e4933a461b34bb08c0f95 - default default] Deleted volume successfully. >2018-08-23 07:03:09.606 505523 INFO cinder.volume.flows.manager.create_volume [req-1e037e3b-f81b-4521-996b-53414df6e415 3ee0feb5e9794db6a80d3130a9382f00 a95eab67288b4726af9c282302c08613 - default default] Volume 14e167db-ba8b-46d6-9363-c4b2c6177928: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-14e167db-ba8b-46d6-9363-c4b2c6177928'} >2018-08-23 07:03:10.143 505523 INFO cinder.volume.flows.manager.create_volume [req-1e037e3b-f81b-4521-996b-53414df6e415 3ee0feb5e9794db6a80d3130a9382f00 a95eab67288b4726af9c282302c08613 - default default] Volume volume-14e167db-ba8b-46d6-9363-c4b2c6177928 (14e167db-ba8b-46d6-9363-c4b2c6177928): created successfully >2018-08-23 07:03:10.146 505523 INFO cinder.volume.manager [req-1e037e3b-f81b-4521-996b-53414df6e415 3ee0feb5e9794db6a80d3130a9382f00 a95eab67288b4726af9c282302c08613 - default default] Created volume successfully. >2018-08-23 07:08:14.823 505523 INFO cinder.volume.manager [req-806bb8eb-fd38-488f-9710-3cab5f9abafe 3ee0feb5e9794db6a80d3130a9382f00 a95eab67288b4726af9c282302c08613 - default default] Deleted volume successfully. >2018-08-23 07:08:20.698 505523 INFO cinder.volume.flows.manager.create_volume [req-d53385d0-4b98-4647-8f73-cccc55d4e2f7 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] Volume 36eb230d-71f6-4094-9b90-1b738e89e108: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-36eb230d-71f6-4094-9b90-1b738e89e108'} >2018-08-23 07:08:21.223 505523 INFO cinder.volume.flows.manager.create_volume [req-d53385d0-4b98-4647-8f73-cccc55d4e2f7 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] Volume volume-36eb230d-71f6-4094-9b90-1b738e89e108 (36eb230d-71f6-4094-9b90-1b738e89e108): created successfully >2018-08-23 07:08:21.227 505523 INFO cinder.volume.manager [req-d53385d0-4b98-4647-8f73-cccc55d4e2f7 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] Created volume successfully. >2018-08-23 07:08:22.903 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a57d3866-1d32-4b41-80da-3e40f9d3af00 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] Unable to get remote copy information for volume volume-36eb230d-71f6-4094-9b90-1b738e89e108 >2018-08-23 07:08:23.186 505523 INFO cinder.volume.manager [req-a57d3866-1d32-4b41-80da-3e40f9d3af00 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] Extend volume completed successfully. >2018-08-23 07:08:25.310 505523 INFO cinder.volume.manager [req-62c4a707-3ae4-4c15-bd69-11bbd109c5ac 569ff43571444e3b87700f208ebf5e68 0b7e237c8b844148be314fe99130935f - default default] Deleted volume successfully. >2018-08-23 07:08:30.157 505523 INFO cinder.volume.flows.manager.create_volume [req-76e4cef2-8faa-45e6-a1b5-d25d089e7f8a 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume 75eeff43-b620-48cb-a0ac-ccc2628e9711: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-75eeff43-b620-48cb-a0ac-ccc2628e9711'} >2018-08-23 07:08:30.682 505523 INFO cinder.volume.flows.manager.create_volume [req-76e4cef2-8faa-45e6-a1b5-d25d089e7f8a 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume volume-75eeff43-b620-48cb-a0ac-ccc2628e9711 (75eeff43-b620-48cb-a0ac-ccc2628e9711): created successfully >2018-08-23 07:08:30.686 505523 INFO cinder.volume.manager [req-76e4cef2-8faa-45e6-a1b5-d25d089e7f8a 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Created volume successfully. >2018-08-23 07:08:33.446 505523 INFO cinder.volume.flows.manager.create_volume [req-f8696b0a-a816-42f8-be54-e112184926ad 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume 90c81dfb-1b42-4dec-9af9-b2baea2bd03c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-90c81dfb-1b42-4dec-9af9-b2baea2bd03c'} >2018-08-23 07:08:34.253 505523 INFO cinder.volume.flows.manager.create_volume [req-f8696b0a-a816-42f8-be54-e112184926ad 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume volume-90c81dfb-1b42-4dec-9af9-b2baea2bd03c (90c81dfb-1b42-4dec-9af9-b2baea2bd03c): created successfully >2018-08-23 07:08:34.257 505523 INFO cinder.volume.manager [req-f8696b0a-a816-42f8-be54-e112184926ad 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Created volume successfully. >2018-08-23 07:08:36.122 505523 INFO cinder.volume.manager [req-40167780-7f73-49e2-9542-80b0b38d500b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Deleted volume successfully. >2018-08-23 07:08:37.609 505523 INFO cinder.volume.manager [req-23a79378-f121-4197-820d-04651acc099a 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Deleted volume successfully. >2018-08-23 07:08:38.226 505523 INFO cinder.volume.flows.manager.create_volume [req-045a9d0b-093d-4611-b081-b5016c8f21c1 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume a334c3c3-8278-4983-8860-5fdd19643944: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a334c3c3-8278-4983-8860-5fdd19643944'} >2018-08-23 07:08:38.792 505523 INFO cinder.volume.flows.manager.create_volume [req-045a9d0b-093d-4611-b081-b5016c8f21c1 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume volume-a334c3c3-8278-4983-8860-5fdd19643944 (a334c3c3-8278-4983-8860-5fdd19643944): created successfully >2018-08-23 07:08:38.796 505523 INFO cinder.volume.manager [req-045a9d0b-093d-4611-b081-b5016c8f21c1 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Created volume successfully. >2018-08-23 07:08:39.614 505523 INFO cinder.volume.flows.manager.create_volume [req-bc39652c-4643-46af-9326-4d92b2d5c2e9 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume 78eed072-5c6a-4800-aa81-9c21b35d4553: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-78eed072-5c6a-4800-aa81-9c21b35d4553', 'source_volid': 'a334c3c3-8278-4983-8860-5fdd19643944', 'source_volstatus': u'available'} >2018-08-23 07:08:41.069 505523 INFO cinder.volume.flows.manager.create_volume [req-bc39652c-4643-46af-9326-4d92b2d5c2e9 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume volume-78eed072-5c6a-4800-aa81-9c21b35d4553 (78eed072-5c6a-4800-aa81-9c21b35d4553): created successfully >2018-08-23 07:08:41.073 505523 INFO cinder.volume.manager [req-bc39652c-4643-46af-9326-4d92b2d5c2e9 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Created volume successfully. >2018-08-23 07:08:42.436 505523 INFO cinder.volume.flows.manager.create_volume [req-20f92082-276e-4de6-b8c7-80dd9dc0ede9 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume 55db2b71-fe65-4c24-81c6-1d0955bfd5c4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-55db2b71-fe65-4c24-81c6-1d0955bfd5c4'} >2018-08-23 07:08:42.967 505523 INFO cinder.volume.flows.manager.create_volume [req-20f92082-276e-4de6-b8c7-80dd9dc0ede9 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume volume-55db2b71-fe65-4c24-81c6-1d0955bfd5c4 (55db2b71-fe65-4c24-81c6-1d0955bfd5c4): created successfully >2018-08-23 07:08:42.971 505523 INFO cinder.volume.manager [req-20f92082-276e-4de6-b8c7-80dd9dc0ede9 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Created volume successfully. >2018-08-23 07:08:44.990 505523 INFO cinder.volume.manager [req-e4972757-5fec-4936-8b52-34d02b3f47ca 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Deleted volume successfully. >2018-08-23 07:08:49.172 505523 INFO cinder.volume.manager [req-e89fb61d-bdc9-436e-8d4c-b8e3d3fc208a 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Deleted volume successfully. >2018-08-23 07:08:50.936 505523 INFO cinder.volume.flows.manager.create_volume [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume 1718c22e-cfe8-4a3b-87ff-e53963612b21: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-1718c22e-cfe8-4a3b-87ff-e53963612b21', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f2695504a90>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 07:08:51.902 505523 INFO cinder.image.image_utils [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-23 07:09:00.431 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] initialize_connection: Did not find a preferred node for volume volume-1718c22e-cfe8-4a3b-87ff-e53963612b21. >2018-08-23 07:09:01.016 505523 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 07:09:05.874 505523 INFO cinder.image.image_utils [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 07:09:07.977 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 07:09:08.334 505523 INFO cinder.volume.flows.manager.create_volume [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume volume-1718c22e-cfe8-4a3b-87ff-e53963612b21 (1718c22e-cfe8-4a3b-87ff-e53963612b21): created successfully >2018-08-23 07:09:08.343 505523 INFO cinder.volume.manager [req-6dede175-4c50-459b-b037-8a9b1be2e95b 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Created volume successfully. >2018-08-23 07:09:10.112 505523 INFO cinder.volume.flows.manager.create_volume [req-ac50eb7c-12ad-4133-8bd7-d51b183fe821 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume cd120e67-c94f-43ac-85de-589afec98710: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cd120e67-c94f-43ac-85de-589afec98710'} >2018-08-23 07:09:10.634 505523 INFO cinder.volume.flows.manager.create_volume [req-ac50eb7c-12ad-4133-8bd7-d51b183fe821 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Volume volume-cd120e67-c94f-43ac-85de-589afec98710 (cd120e67-c94f-43ac-85de-589afec98710): created successfully >2018-08-23 07:09:10.638 505523 INFO cinder.volume.manager [req-ac50eb7c-12ad-4133-8bd7-d51b183fe821 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Created volume successfully. >2018-08-23 07:09:12.380 505523 INFO cinder.volume.manager [req-9ac53b5b-b037-4a51-965f-ee84660dcff1 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Deleted volume successfully. >2018-08-23 07:09:13.584 505523 INFO cinder.volume.manager [req-df2124f7-11af-4d5c-b38b-f9569a718ba8 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Deleted volume successfully. >2018-08-23 07:09:14.806 505523 INFO cinder.volume.manager [req-5b7a1b44-7746-4bcd-a2b1-b68b8b55d396 2dfc21d1f6914f8b9cf65b4c76ff6c08 37a56f7355fc402da8e499e5c7a6c0ba - default default] Deleted volume successfully. >2018-08-23 07:09:23.818 505523 INFO cinder.volume.flows.manager.create_volume [req-c93be2fa-a77d-43f8-aa9b-84c6a5156f3c e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Volume 96ba84b0-c68e-4f20-8e9a-83b196f491f1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-96ba84b0-c68e-4f20-8e9a-83b196f491f1'} >2018-08-23 07:09:24.353 505523 INFO cinder.volume.flows.manager.create_volume [req-c93be2fa-a77d-43f8-aa9b-84c6a5156f3c e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Volume volume-96ba84b0-c68e-4f20-8e9a-83b196f491f1 (96ba84b0-c68e-4f20-8e9a-83b196f491f1): created successfully >2018-08-23 07:09:24.357 505523 INFO cinder.volume.manager [req-c93be2fa-a77d-43f8-aa9b-84c6a5156f3c e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Created volume successfully. >2018-08-23 07:09:25.842 505523 INFO cinder.volume.flows.manager.create_volume [req-e23e647b-baea-486d-9cb0-97d680b05085 e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Volume 3158f1a0-293f-4aac-ac5a-7c325308ce1e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3158f1a0-293f-4aac-ac5a-7c325308ce1e'} >2018-08-23 07:09:26.378 505523 INFO cinder.volume.flows.manager.create_volume [req-e23e647b-baea-486d-9cb0-97d680b05085 e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Volume volume-3158f1a0-293f-4aac-ac5a-7c325308ce1e (3158f1a0-293f-4aac-ac5a-7c325308ce1e): created successfully >2018-08-23 07:09:26.382 505523 INFO cinder.volume.manager [req-e23e647b-baea-486d-9cb0-97d680b05085 e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Created volume successfully. >2018-08-23 07:09:27.947 505464 INFO cinder.volume.flows.manager.create_volume [req-fd864784-aace-42da-a6f1-d1e43fd28ccc e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Volume 2f464bd5-2cea-4eed-9a51-032cd1d39a55: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2f464bd5-2cea-4eed-9a51-032cd1d39a55'} >2018-08-23 07:09:28.540 505464 INFO cinder.volume.flows.manager.create_volume [req-fd864784-aace-42da-a6f1-d1e43fd28ccc e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Volume volume-2f464bd5-2cea-4eed-9a51-032cd1d39a55 (2f464bd5-2cea-4eed-9a51-032cd1d39a55): created successfully >2018-08-23 07:09:28.544 505464 INFO cinder.volume.manager [req-fd864784-aace-42da-a6f1-d1e43fd28ccc e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Created volume successfully. >2018-08-23 07:09:34.265 505523 INFO cinder.volume.manager [req-2578223f-a6e1-4421-99ca-773ffba9a133 e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Deleted volume successfully. >2018-08-23 07:09:34.310 505523 INFO cinder.volume.manager [req-94a3a6cc-7862-4dde-8f5f-cf7176048624 e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Deleted volume successfully. >2018-08-23 07:09:34.509 505464 INFO cinder.volume.manager [req-0ac8718b-145b-4918-a739-891b51768f7f e2cb95405de548109722fbb8eea87d64 d841e1d4301043ceba18ed4393e9c6f8 - default default] Deleted volume successfully. >2018-08-23 07:09:38.749 505523 INFO cinder.volume.flows.manager.create_volume [req-e3b6fd33-d2d3-4198-8693-73b8aba9e6fe c8ca3c4e1cf94a74ad533ea4c6ddc097 3b73f5d9bebb4ea49a80bd661c7734ff - default default] Volume ea3af7e5-fdc8-45f7-b1f7-bb2447db6965: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ea3af7e5-fdc8-45f7-b1f7-bb2447db6965'} >2018-08-23 07:09:39.295 505523 INFO cinder.volume.flows.manager.create_volume [req-e3b6fd33-d2d3-4198-8693-73b8aba9e6fe c8ca3c4e1cf94a74ad533ea4c6ddc097 3b73f5d9bebb4ea49a80bd661c7734ff - default default] Volume volume-ea3af7e5-fdc8-45f7-b1f7-bb2447db6965 (ea3af7e5-fdc8-45f7-b1f7-bb2447db6965): created successfully >2018-08-23 07:09:39.298 505523 INFO cinder.volume.manager [req-e3b6fd33-d2d3-4198-8693-73b8aba9e6fe c8ca3c4e1cf94a74ad533ea4c6ddc097 3b73f5d9bebb4ea49a80bd661c7734ff - default default] Created volume successfully. >2018-08-23 07:14:55.624 505523 INFO cinder.volume.manager [req-3901b7a5-ca95-4ff1-bd09-794221d902d5 c8ca3c4e1cf94a74ad533ea4c6ddc097 3b73f5d9bebb4ea49a80bd661c7734ff - default default] Deleted volume successfully. >2018-08-23 07:14:59.832 505523 INFO cinder.volume.flows.manager.create_volume [req-71726c8f-2005-41f7-9492-5b2458026ea8 18ff39835a534a92986072f252398a57 781484c7bd5941d7a65d5335c8782988 - default default] Volume 591dd437-efee-4285-a22d-a0a9142466cf: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-591dd437-efee-4285-a22d-a0a9142466cf'} >2018-08-23 07:15:00.358 505523 INFO cinder.volume.flows.manager.create_volume [req-71726c8f-2005-41f7-9492-5b2458026ea8 18ff39835a534a92986072f252398a57 781484c7bd5941d7a65d5335c8782988 - default default] Volume volume-591dd437-efee-4285-a22d-a0a9142466cf (591dd437-efee-4285-a22d-a0a9142466cf): created successfully >2018-08-23 07:15:00.362 505523 INFO cinder.volume.manager [req-71726c8f-2005-41f7-9492-5b2458026ea8 18ff39835a534a92986072f252398a57 781484c7bd5941d7a65d5335c8782988 - default default] Created volume successfully. >2018-08-23 07:15:03.166 505523 INFO cinder.volume.manager [req-6c795a3e-9bd8-4452-a6f0-8651beea04b1 18ff39835a534a92986072f252398a57 781484c7bd5941d7a65d5335c8782988 - default default] Deleted volume successfully. >2018-08-23 07:18:34.093 505523 INFO cinder.volume.manager [req-65c5cb42-8a3d-4167-81b2-659706bf8b3e 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-805248882: creating >2018-08-23 07:18:34.121 505523 INFO cinder.volume.manager [req-65c5cb42-8a3d-4167-81b2-659706bf8b3e 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-805248882: created successfully >2018-08-23 07:18:34.142 505523 INFO cinder.volume.manager [req-65c5cb42-8a3d-4167-81b2-659706bf8b3e 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Create group completed successfully. >2018-08-23 07:18:34.902 505523 INFO cinder.volume.flows.manager.create_volume [req-25d7d5a4-5e1a-4dd5-8759-af19b2199bc4 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume 8b87ca1d-f01e-450a-b74c-6fb42f2696fe: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8b87ca1d-f01e-450a-b74c-6fb42f2696fe'} >2018-08-23 07:18:35.455 505523 INFO cinder.volume.flows.manager.create_volume [req-25d7d5a4-5e1a-4dd5-8759-af19b2199bc4 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume volume-8b87ca1d-f01e-450a-b74c-6fb42f2696fe (8b87ca1d-f01e-450a-b74c-6fb42f2696fe): created successfully >2018-08-23 07:18:35.459 505523 INFO cinder.volume.manager [req-25d7d5a4-5e1a-4dd5-8759-af19b2199bc4 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Created volume successfully. >2018-08-23 07:18:36.695 505523 INFO cinder.volume.manager [req-c2cf66ec-cdd2-4b6b-9fe6-ccc685e1eaae 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] GroupSnapshot 31b2051b-9fe2-469b-81a8-a4b2106db67b: creating. >2018-08-23 07:18:38.702 505523 INFO cinder.volume.manager [req-c2cf66ec-cdd2-4b6b-9fe6-ccc685e1eaae 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] group_snapshot 31b2051b-9fe2-469b-81a8-a4b2106db67b: created successfully >2018-08-23 07:18:39.687 505523 INFO cinder.volume.manager [req-f16ae1c2-bd54-40eb-be34-cc9e7e5061f2 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] group_snapshot 31b2051b-9fe2-469b-81a8-a4b2106db67b: deleting >2018-08-23 07:18:41.719 505523 WARNING cinder.quota [req-f16ae1c2-bd54-40eb-be34-cc9e7e5061f2 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-958030506 is set by the default quota flag: quota_snapshots_tempest-volume-type-958030506, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:18:41.749 505523 INFO cinder.volume.manager [req-f16ae1c2-bd54-40eb-be34-cc9e7e5061f2 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] group_snapshot 31b2051b-9fe2-469b-81a8-a4b2106db67b: deleted successfully >2018-08-23 07:18:43.664 505523 WARNING cinder.quota [req-3b75d802-660b-480b-9edb-80afc56ff371 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-958030506 is set by the default quota flag: quota_gigabytes_tempest-volume-type-958030506, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:18:43.665 505523 WARNING cinder.quota [req-3b75d802-660b-480b-9edb-80afc56ff371 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-958030506 is set by the default quota flag: quota_volumes_tempest-volume-type-958030506, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:18:43.937 505523 INFO cinder.volume.manager [req-3b75d802-660b-480b-9edb-80afc56ff371 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Delete group completed successfully. >2018-08-23 07:18:44.326 505523 INFO cinder.volume.manager [req-cbbc3309-797d-46b1-91b3-331f7f815582 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-1039344743: creating >2018-08-23 07:18:44.347 505523 INFO cinder.volume.manager [req-cbbc3309-797d-46b1-91b3-331f7f815582 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-1039344743: created successfully >2018-08-23 07:18:44.369 505523 INFO cinder.volume.manager [req-cbbc3309-797d-46b1-91b3-331f7f815582 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Create group completed successfully. >2018-08-23 07:18:44.970 505523 INFO cinder.volume.flows.manager.create_volume [req-eac43d3e-61d6-47e7-833e-f3a457d20527 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume be1bba03-d8ae-4c31-8567-47fb15b8a0bf: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-be1bba03-d8ae-4c31-8567-47fb15b8a0bf'} >2018-08-23 07:18:45.513 505523 INFO cinder.volume.flows.manager.create_volume [req-eac43d3e-61d6-47e7-833e-f3a457d20527 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume volume-be1bba03-d8ae-4c31-8567-47fb15b8a0bf (be1bba03-d8ae-4c31-8567-47fb15b8a0bf): created successfully >2018-08-23 07:18:45.517 505523 INFO cinder.volume.manager [req-eac43d3e-61d6-47e7-833e-f3a457d20527 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Created volume successfully. >2018-08-23 07:18:47.178 505523 WARNING cinder.quota [req-f23dc5e9-da09-4190-bb43-b7b2356c346d 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-363601534 is set by the default quota flag: quota_gigabytes_tempest-volume-type-363601534, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:18:47.179 505523 WARNING cinder.quota [req-f23dc5e9-da09-4190-bb43-b7b2356c346d 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-363601534 is set by the default quota flag: quota_volumes_tempest-volume-type-363601534, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:18:47.462 505523 INFO cinder.volume.manager [req-f23dc5e9-da09-4190-bb43-b7b2356c346d 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Delete group completed successfully. >2018-08-23 07:18:47.745 505523 INFO cinder.volume.manager [req-e75ff629-36c4-4fe6-a957-e478755d2a7b 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-734556151: creating >2018-08-23 07:18:47.767 505523 INFO cinder.volume.manager [req-e75ff629-36c4-4fe6-a957-e478755d2a7b 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-734556151: created successfully >2018-08-23 07:18:47.788 505523 INFO cinder.volume.manager [req-e75ff629-36c4-4fe6-a957-e478755d2a7b 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Create group completed successfully. >2018-08-23 07:18:48.171 505523 INFO cinder.volume.flows.manager.create_volume [req-44e976d9-c6de-47a5-8748-b0a2e505a99c 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume 3cdcd151-12e0-4b71-bd04-54040dff588a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3cdcd151-12e0-4b71-bd04-54040dff588a'} >2018-08-23 07:18:48.825 505523 INFO cinder.volume.flows.manager.create_volume [req-44e976d9-c6de-47a5-8748-b0a2e505a99c 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume volume-3cdcd151-12e0-4b71-bd04-54040dff588a (3cdcd151-12e0-4b71-bd04-54040dff588a): created successfully >2018-08-23 07:18:48.829 505523 INFO cinder.volume.manager [req-44e976d9-c6de-47a5-8748-b0a2e505a99c 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Created volume successfully. >2018-08-23 07:18:49.446 505523 INFO cinder.volume.manager [req-7b327351-6e81-449f-94bb-76ac8762ea59 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] GroupSnapshot a32f89e9-ff6f-4145-817b-a9306d4a46c4: creating. >2018-08-23 07:18:54.222 505523 INFO cinder.volume.manager [req-7b327351-6e81-449f-94bb-76ac8762ea59 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] group_snapshot a32f89e9-ff6f-4145-817b-a9306d4a46c4: created successfully >2018-08-23 07:18:58.166 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3403a371-c7dd-4b7b-99a2-4682652589ea 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Updating status for CG: 5edee4ef-deac-466b-a0e6-3b40ddb4dd02. >2018-08-23 07:18:58.278 505523 INFO cinder.volume.manager [req-3403a371-c7dd-4b7b-99a2-4682652589ea 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Create group from source-snapshot-a32f89e9-ff6f-4145-817b-a9306d4a46c4 completed successfully. >2018-08-23 07:19:02.799 505523 WARNING cinder.quota [req-1b8c02f2-1777-4d9e-a24b-b4882d107b81 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1486560668 is set by the default quota flag: quota_volumes_tempest-volume-type-1486560668, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:19:02.800 505523 WARNING cinder.quota [req-1b8c02f2-1777-4d9e-a24b-b4882d107b81 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1486560668 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1486560668, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:19:03.071 505523 INFO cinder.volume.manager [req-1b8c02f2-1777-4d9e-a24b-b4882d107b81 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Delete group completed successfully. >2018-08-23 07:19:03.524 505523 INFO cinder.volume.manager [req-5d714531-5d89-440e-b556-ae7ede2f68e7 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] group_snapshot a32f89e9-ff6f-4145-817b-a9306d4a46c4: deleting >2018-08-23 07:19:05.542 505523 WARNING cinder.quota [req-5d714531-5d89-440e-b556-ae7ede2f68e7 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-1486560668 is set by the default quota flag: quota_snapshots_tempest-volume-type-1486560668, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:19:05.574 505523 INFO cinder.volume.manager [req-5d714531-5d89-440e-b556-ae7ede2f68e7 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] group_snapshot a32f89e9-ff6f-4145-817b-a9306d4a46c4: deleted successfully >2018-08-23 07:19:07.278 505523 INFO cinder.volume.manager [req-6e37e585-4729-4fc6-a1b5-a73b2f96ec99 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Delete group completed successfully. >2018-08-23 07:19:07.859 505523 INFO cinder.volume.manager [req-b71e2d86-6892-4b4f-827f-3aa8ca9332d8 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-1660255899: creating >2018-08-23 07:19:07.880 505523 INFO cinder.volume.manager [req-b71e2d86-6892-4b4f-827f-3aa8ca9332d8 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Group tempest-CG-1660255899: created successfully >2018-08-23 07:19:07.900 505523 INFO cinder.volume.manager [req-b71e2d86-6892-4b4f-827f-3aa8ca9332d8 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Create group completed successfully. >2018-08-23 07:19:08.293 505523 INFO cinder.volume.flows.manager.create_volume [req-7ac8a199-4d6f-46bc-9ff9-ae10f6eea47a 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume 378beef1-85a5-4144-bab6-02d3f9a87e04: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-378beef1-85a5-4144-bab6-02d3f9a87e04'} >2018-08-23 07:19:08.843 505523 INFO cinder.volume.flows.manager.create_volume [req-7ac8a199-4d6f-46bc-9ff9-ae10f6eea47a 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Volume volume-378beef1-85a5-4144-bab6-02d3f9a87e04 (378beef1-85a5-4144-bab6-02d3f9a87e04): created successfully >2018-08-23 07:19:08.847 505523 INFO cinder.volume.manager [req-7ac8a199-4d6f-46bc-9ff9-ae10f6eea47a 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Created volume successfully. >2018-08-23 07:19:11.667 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1e4e4d84-4bc0-4e55-8328-a2a2ebc368cf 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Updating status for CG: 2a0cee28-842a-4fb6-b7ec-d39223076b03. >2018-08-23 07:19:11.872 505523 INFO cinder.volume.manager [req-1e4e4d84-4bc0-4e55-8328-a2a2ebc368cf 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Create group from source-group-1cf68496-7675-4cb8-a1bb-7b3b209f6043 completed successfully. >2018-08-23 07:19:16.257 505523 WARNING cinder.quota [req-dc33088d-c04a-43a1-9345-9c1735490e28 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1683297232 is set by the default quota flag: quota_volumes_tempest-volume-type-1683297232, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:19:16.258 505523 WARNING cinder.quota [req-dc33088d-c04a-43a1-9345-9c1735490e28 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1683297232 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1683297232, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:19:16.545 505523 INFO cinder.volume.manager [req-dc33088d-c04a-43a1-9345-9c1735490e28 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Delete group completed successfully. >2018-08-23 07:19:18.250 505523 INFO cinder.volume.manager [req-e405cc18-d44c-4f57-a00b-b9c124d6e6f9 93a770784c444d86881bfe132b0a0dd3 05e528090ea04408806b085708c80684 - default default] Delete group completed successfully. >2018-08-23 07:19:57.003 505523 INFO cinder.volume.flows.manager.create_volume [req-b51adfda-69f8-4901-ac15-4af566caac67 271e431c6d5f4e30bd86894912e0bd4a 85dfb2d3df9f40a383ad791557a9f90b - default default] Volume 58a5b8dd-d7c8-44b0-bca9-1cfc8f28726a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-58a5b8dd-d7c8-44b0-bca9-1cfc8f28726a'} >2018-08-23 07:19:57.535 505523 INFO cinder.volume.flows.manager.create_volume [req-b51adfda-69f8-4901-ac15-4af566caac67 271e431c6d5f4e30bd86894912e0bd4a 85dfb2d3df9f40a383ad791557a9f90b - default default] Volume volume-58a5b8dd-d7c8-44b0-bca9-1cfc8f28726a (58a5b8dd-d7c8-44b0-bca9-1cfc8f28726a): created successfully >2018-08-23 07:19:57.538 505523 INFO cinder.volume.manager [req-b51adfda-69f8-4901-ac15-4af566caac67 271e431c6d5f4e30bd86894912e0bd4a 85dfb2d3df9f40a383ad791557a9f90b - default default] Created volume successfully. >2018-08-23 07:19:59.904 505523 INFO cinder.volume.manager [req-dafa25bc-96c9-47a9-8c93-d9f3865a35c6 271e431c6d5f4e30bd86894912e0bd4a 85dfb2d3df9f40a383ad791557a9f90b - default default] Deleted volume successfully. >2018-08-23 07:20:09.447 505523 INFO cinder.volume.flows.manager.create_volume [req-4e70a4d4-1b83-4b4c-b0d2-b540cf4df306 7bb9ef2941414da4a746c798aa011ee5 52440b624ba64b589dcd73385b425008 - default default] Volume 1a552bdd-bd19-40a2-be18-4db838554f74: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1a552bdd-bd19-40a2-be18-4db838554f74'} >2018-08-23 07:20:09.987 505523 INFO cinder.volume.flows.manager.create_volume [req-4e70a4d4-1b83-4b4c-b0d2-b540cf4df306 7bb9ef2941414da4a746c798aa011ee5 52440b624ba64b589dcd73385b425008 - default default] Volume volume-1a552bdd-bd19-40a2-be18-4db838554f74 (1a552bdd-bd19-40a2-be18-4db838554f74): created successfully >2018-08-23 07:20:09.991 505523 INFO cinder.volume.manager [req-4e70a4d4-1b83-4b4c-b0d2-b540cf4df306 7bb9ef2941414da4a746c798aa011ee5 52440b624ba64b589dcd73385b425008 - default default] Created volume successfully. >2018-08-23 07:20:13.181 505523 WARNING cinder.quota [req-1ec37796-d2de-4bb3-89b5-c45ab24f24f1 7bb9ef2941414da4a746c798aa011ee5 52440b624ba64b589dcd73385b425008 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1704485059 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1704485059, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:20:13.182 505523 WARNING cinder.quota [req-1ec37796-d2de-4bb3-89b5-c45ab24f24f1 7bb9ef2941414da4a746c798aa011ee5 52440b624ba64b589dcd73385b425008 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1704485059 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1704485059, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:20:13.433 505523 INFO cinder.volume.manager [req-1ec37796-d2de-4bb3-89b5-c45ab24f24f1 7bb9ef2941414da4a746c798aa011ee5 52440b624ba64b589dcd73385b425008 - default default] Deleted volume successfully. >2018-08-23 07:20:22.165 505523 INFO cinder.volume.flows.manager.create_volume [req-db557e87-76c0-4a18-8c36-636cb5e8e2aa d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] Volume fb2fbb3a-b678-4941-b659-a56b8a039fa0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fb2fbb3a-b678-4941-b659-a56b8a039fa0'} >2018-08-23 07:20:22.726 505523 INFO cinder.volume.flows.manager.create_volume [req-db557e87-76c0-4a18-8c36-636cb5e8e2aa d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] Volume volume-fb2fbb3a-b678-4941-b659-a56b8a039fa0 (fb2fbb3a-b678-4941-b659-a56b8a039fa0): created successfully >2018-08-23 07:20:22.730 505523 INFO cinder.volume.manager [req-db557e87-76c0-4a18-8c36-636cb5e8e2aa d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] Created volume successfully. >2018-08-23 07:20:24.315 505523 INFO cinder.volume.manager [req-cd26ae82-ab44-4184-8f13-c75b315c7154 d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] Retype volume completed successfully. >2018-08-23 07:20:28.990 505523 WARNING cinder.quota [req-318de36f-fb70-44c0-8ea4-afbeaabe943c d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-945713087 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-945713087, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:20:28.991 505523 WARNING cinder.quota [req-318de36f-fb70-44c0-8ea4-afbeaabe943c d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-945713087 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-945713087, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:20:29.241 505523 INFO cinder.volume.manager [req-318de36f-fb70-44c0-8ea4-afbeaabe943c d59dcfc9645e48d09de39a69a75b1490 74932d17296640a1afe3a8a27e23253f - default default] Deleted volume successfully. >2018-08-23 07:26:05.192 505523 INFO cinder.volume.flows.manager.create_volume [req-da444ec7-1a85-465f-8825-d47a3673a1c8 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume 24aae2a9-cc47-48cc-b9d1-f56d8bc5bb09: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-24aae2a9-cc47-48cc-b9d1-f56d8bc5bb09'} >2018-08-23 07:26:05.751 505523 INFO cinder.volume.flows.manager.create_volume [req-da444ec7-1a85-465f-8825-d47a3673a1c8 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume volume-24aae2a9-cc47-48cc-b9d1-f56d8bc5bb09 (24aae2a9-cc47-48cc-b9d1-f56d8bc5bb09): created successfully >2018-08-23 07:26:05.754 505523 INFO cinder.volume.manager [req-da444ec7-1a85-465f-8825-d47a3673a1c8 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Created volume successfully. >2018-08-23 07:26:08.599 505523 INFO cinder.volume.manager [req-e633c391-cd32-41c1-a751-a51c4eb357b2 fd8549cb45a147d6988e48bc1504938d fa7a4ca9ed434c458cc6c65c272d2962 - default default] Deleted volume successfully. >2018-08-23 07:26:09.071 505523 INFO cinder.volume.flows.manager.create_volume [req-1de97e0e-ce40-4fdc-807b-5b79178cd065 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume 34c1c360-8a78-4cea-8c47-ed9af5d18fe6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-34c1c360-8a78-4cea-8c47-ed9af5d18fe6'} >2018-08-23 07:26:09.607 505523 INFO cinder.volume.flows.manager.create_volume [req-1de97e0e-ce40-4fdc-807b-5b79178cd065 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume volume-34c1c360-8a78-4cea-8c47-ed9af5d18fe6 (34c1c360-8a78-4cea-8c47-ed9af5d18fe6): created successfully >2018-08-23 07:26:09.611 505523 INFO cinder.volume.manager [req-1de97e0e-ce40-4fdc-807b-5b79178cd065 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Created volume successfully. >2018-08-23 07:26:12.483 505523 INFO cinder.volume.manager [req-a359d121-ae36-4c68-8715-740de8037e26 fd8549cb45a147d6988e48bc1504938d fa7a4ca9ed434c458cc6c65c272d2962 - default default] Deleted volume successfully. >2018-08-23 07:26:13.272 505523 INFO cinder.volume.flows.manager.create_volume [req-6e28c652-d484-4cba-b818-35b4221cb663 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume 1eb61bb5-9e14-4bf2-a22a-a04f6725b5f0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1eb61bb5-9e14-4bf2-a22a-a04f6725b5f0'} >2018-08-23 07:26:13.806 505523 INFO cinder.volume.flows.manager.create_volume [req-6e28c652-d484-4cba-b818-35b4221cb663 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume volume-1eb61bb5-9e14-4bf2-a22a-a04f6725b5f0 (1eb61bb5-9e14-4bf2-a22a-a04f6725b5f0): created successfully >2018-08-23 07:26:13.810 505523 INFO cinder.volume.manager [req-6e28c652-d484-4cba-b818-35b4221cb663 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Created volume successfully. >2018-08-23 07:26:16.033 505523 INFO cinder.volume.manager [req-f04fab6b-174c-4b67-81df-dc8a08618d00 fd8549cb45a147d6988e48bc1504938d fa7a4ca9ed434c458cc6c65c272d2962 - default default] Deleted volume successfully. >2018-08-23 07:26:16.405 505523 INFO cinder.volume.flows.manager.create_volume [req-1a96f345-7168-4f1d-8258-bd88d57f57e3 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume 6749602e-57a3-4c36-982a-af75cb56d21e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6749602e-57a3-4c36-982a-af75cb56d21e'} >2018-08-23 07:26:17.033 505523 INFO cinder.volume.flows.manager.create_volume [req-1a96f345-7168-4f1d-8258-bd88d57f57e3 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume volume-6749602e-57a3-4c36-982a-af75cb56d21e (6749602e-57a3-4c36-982a-af75cb56d21e): created successfully >2018-08-23 07:26:17.036 505523 INFO cinder.volume.manager [req-1a96f345-7168-4f1d-8258-bd88d57f57e3 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Created volume successfully. >2018-08-23 07:26:19.538 505523 INFO cinder.volume.manager [req-98e4cfc0-f092-4a3d-8e20-b6be7987328b fd8549cb45a147d6988e48bc1504938d fa7a4ca9ed434c458cc6c65c272d2962 - default default] Deleted volume successfully. >2018-08-23 07:26:19.888 505523 INFO cinder.volume.flows.manager.create_volume [req-92d1b5c8-1a93-484e-a080-d0a39a325e71 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume 790438ad-d5bd-4009-a108-a5c792099eab: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-790438ad-d5bd-4009-a108-a5c792099eab'} >2018-08-23 07:26:20.412 505523 INFO cinder.volume.flows.manager.create_volume [req-92d1b5c8-1a93-484e-a080-d0a39a325e71 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Volume volume-790438ad-d5bd-4009-a108-a5c792099eab (790438ad-d5bd-4009-a108-a5c792099eab): created successfully >2018-08-23 07:26:20.416 505523 INFO cinder.volume.manager [req-92d1b5c8-1a93-484e-a080-d0a39a325e71 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Created volume successfully. >2018-08-23 07:26:23.129 505523 INFO cinder.volume.manager [req-97b15369-69c5-44a5-85a3-8ae9c57e4048 77425ef8b3d04663b0ea80e97b5631f9 10f8cb22f66b461a9416c6b34743cd01 - default default] Deleted volume successfully. >2018-08-23 07:26:30.657 505523 INFO cinder.volume.flows.manager.create_volume [req-0c4fe5bd-9e3f-4082-9680-d34886dea998 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Volume 769a4b03-548d-4960-a193-a993cd4a9f75: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-769a4b03-548d-4960-a193-a993cd4a9f75'} >2018-08-23 07:26:31.180 505523 INFO cinder.volume.flows.manager.create_volume [req-0c4fe5bd-9e3f-4082-9680-d34886dea998 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Volume volume-769a4b03-548d-4960-a193-a993cd4a9f75 (769a4b03-548d-4960-a193-a993cd4a9f75): created successfully >2018-08-23 07:26:31.184 505523 INFO cinder.volume.manager [req-0c4fe5bd-9e3f-4082-9680-d34886dea998 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Created volume successfully. >2018-08-23 07:26:32.584 505464 INFO cinder.volume.flows.manager.create_volume [req-3cc60711-5b1b-47bf-89c8-8cfa70e6aa20 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Volume 0b1b5482-1e3b-483a-9cb8-785b48305c32: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0b1b5482-1e3b-483a-9cb8-785b48305c32'} >2018-08-23 07:26:33.174 505464 INFO cinder.volume.flows.manager.create_volume [req-3cc60711-5b1b-47bf-89c8-8cfa70e6aa20 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Volume volume-0b1b5482-1e3b-483a-9cb8-785b48305c32 (0b1b5482-1e3b-483a-9cb8-785b48305c32): created successfully >2018-08-23 07:26:33.179 505464 INFO cinder.volume.manager [req-3cc60711-5b1b-47bf-89c8-8cfa70e6aa20 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Created volume successfully. >2018-08-23 07:26:33.990 505523 INFO cinder.volume.flows.manager.create_volume [req-f97cf699-0e52-4889-a8ec-3b8297443ca1 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Volume 1ab166d6-16b2-4370-b0c4-2698c2538c0c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1ab166d6-16b2-4370-b0c4-2698c2538c0c'} >2018-08-23 07:26:34.624 505523 INFO cinder.volume.flows.manager.create_volume [req-f97cf699-0e52-4889-a8ec-3b8297443ca1 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Volume volume-1ab166d6-16b2-4370-b0c4-2698c2538c0c (1ab166d6-16b2-4370-b0c4-2698c2538c0c): created successfully >2018-08-23 07:26:34.628 505523 INFO cinder.volume.manager [req-f97cf699-0e52-4889-a8ec-3b8297443ca1 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Created volume successfully. >2018-08-23 07:26:36.054 505464 INFO cinder.volume.flows.manager.create_volume [req-67a22fdd-4852-4a47-aab8-7351a625e544 011d7ca2e8634060be74afe39016e5a4 91a28f7105d84e6db0dc525b64b75ea9 - default default] Volume 00104f75-aff0-4f75-9854-bf9c48cc9574: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-00104f75-aff0-4f75-9854-bf9c48cc9574'} >2018-08-23 07:26:36.645 505464 INFO cinder.volume.flows.manager.create_volume [req-67a22fdd-4852-4a47-aab8-7351a625e544 011d7ca2e8634060be74afe39016e5a4 91a28f7105d84e6db0dc525b64b75ea9 - default default] Volume volume-00104f75-aff0-4f75-9854-bf9c48cc9574 (00104f75-aff0-4f75-9854-bf9c48cc9574): created successfully >2018-08-23 07:26:36.649 505464 INFO cinder.volume.manager [req-67a22fdd-4852-4a47-aab8-7351a625e544 011d7ca2e8634060be74afe39016e5a4 91a28f7105d84e6db0dc525b64b75ea9 - default default] Created volume successfully. >2018-08-23 07:26:38.284 505464 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 07:26:38.364 505464 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 07:26:41.081 505464 INFO cinder.volume.manager [req-b0fb1173-9e4f-42c3-b4a2-024d709ecbb1 011d7ca2e8634060be74afe39016e5a4 91a28f7105d84e6db0dc525b64b75ea9 - default default] Deleted volume successfully. >2018-08-23 07:26:41.105 505523 INFO cinder.volume.manager [req-9969b97c-af44-43ac-940e-e6bbcae0558b 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Deleted volume successfully. >2018-08-23 07:26:41.169 505464 INFO cinder.volume.manager [req-c9f6fe41-edd1-41bd-aed8-9f6ffe8e78b8 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Deleted volume successfully. >2018-08-23 07:26:41.355 505523 INFO cinder.volume.manager [req-ed6050c7-0cbf-4d05-a80f-969bd886cfff 6f06b70904954af2bf007f6b59743529 3a0b43d402c5416898c61b70cc811140 - default default] Deleted volume successfully. >2018-08-23 07:26:47.755 505523 INFO cinder.volume.flows.manager.create_volume [req-8e42720c-30da-4fc0-b3c6-516f00c456c0 ff2bccfc1438465fbd7cf07c54528487 6022834077d54c51872dc8f4364c58e0 - default default] Volume ae3eac3d-0573-4853-ae9f-dd6360ee97d1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ae3eac3d-0573-4853-ae9f-dd6360ee97d1'} >2018-08-23 07:26:48.314 505523 INFO cinder.volume.flows.manager.create_volume [req-8e42720c-30da-4fc0-b3c6-516f00c456c0 ff2bccfc1438465fbd7cf07c54528487 6022834077d54c51872dc8f4364c58e0 - default default] Volume volume-ae3eac3d-0573-4853-ae9f-dd6360ee97d1 (ae3eac3d-0573-4853-ae9f-dd6360ee97d1): created successfully >2018-08-23 07:26:48.317 505523 INFO cinder.volume.manager [req-8e42720c-30da-4fc0-b3c6-516f00c456c0 ff2bccfc1438465fbd7cf07c54528487 6022834077d54c51872dc8f4364c58e0 - default default] Created volume successfully. >2018-08-23 07:26:53.448 505523 INFO cinder.volume.manager [req-ea278a81-013c-4865-a01f-3790be3f5dc2 ff2bccfc1438465fbd7cf07c54528487 6022834077d54c51872dc8f4364c58e0 - default default] Deleted volume successfully. >2018-08-23 07:27:01.532 505523 INFO cinder.volume.flows.manager.create_volume [req-8f00ff00-6a85-47f6-abfe-391b5c05fd83 fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] Volume fa0303ef-933d-4b88-a56c-e58d0eedee46: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fa0303ef-933d-4b88-a56c-e58d0eedee46'} >2018-08-23 07:27:03.596 505523 INFO cinder.volume.flows.manager.create_volume [req-8f00ff00-6a85-47f6-abfe-391b5c05fd83 fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] Volume volume-fa0303ef-933d-4b88-a56c-e58d0eedee46 (fa0303ef-933d-4b88-a56c-e58d0eedee46): created successfully >2018-08-23 07:27:03.600 505523 INFO cinder.volume.manager [req-8f00ff00-6a85-47f6-abfe-391b5c05fd83 fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] Created volume successfully. >2018-08-23 07:27:05.103 505523 INFO cinder.volume.manager [req-d7913e61-d675-40cc-9bac-ba0b2011389c 0adbfc8a2a90455e854f1d5691da3cf9 6e60ec3b0a16418a816c5cdf3b3b3460 - default default] Transfer volume completed successfully. >2018-08-23 07:27:07.049 505523 INFO cinder.volume.manager [req-0850713e-e240-4552-af89-69ba641db0ac 1ac3f6d51a164e36bd68d101312ff434 d9d2cce4315a445b8f5b2f79bf519090 - default default] Deleted volume successfully. >2018-08-23 07:27:07.985 505523 INFO cinder.volume.flows.manager.create_volume [req-4ef4fce5-8289-45bb-af16-b4ffa389d60c fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] Volume c5ccef51-d9c5-4c47-b37a-53c22fb56e79: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c5ccef51-d9c5-4c47-b37a-53c22fb56e79'} >2018-08-23 07:27:08.606 505523 INFO cinder.volume.flows.manager.create_volume [req-4ef4fce5-8289-45bb-af16-b4ffa389d60c fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] Volume volume-c5ccef51-d9c5-4c47-b37a-53c22fb56e79 (c5ccef51-d9c5-4c47-b37a-53c22fb56e79): created successfully >2018-08-23 07:27:08.610 505523 INFO cinder.volume.manager [req-4ef4fce5-8289-45bb-af16-b4ffa389d60c fd99ae8e6513497eba8131f992b86d45 de2ac2aacbf54466ace4930207c01a48 - default default] Created volume successfully. >2018-08-23 07:27:11.422 505523 INFO cinder.volume.manager [req-df047236-54b9-4c8a-bee3-20842f4c7b60 1ac3f6d51a164e36bd68d101312ff434 d9d2cce4315a445b8f5b2f79bf519090 - default default] Deleted volume successfully. >2018-08-23 07:27:18.076 505523 INFO cinder.volume.flows.manager.create_volume [req-85d28709-edcb-4a39-936d-c4cf79d15947 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] Volume 244c589e-3a0c-431a-b8f4-39c356a7bcc4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-244c589e-3a0c-431a-b8f4-39c356a7bcc4'} >2018-08-23 07:27:18.606 505523 INFO cinder.volume.flows.manager.create_volume [req-85d28709-edcb-4a39-936d-c4cf79d15947 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] Volume volume-244c589e-3a0c-431a-b8f4-39c356a7bcc4 (244c589e-3a0c-431a-b8f4-39c356a7bcc4): created successfully >2018-08-23 07:27:18.611 505523 INFO cinder.volume.manager [req-85d28709-edcb-4a39-936d-c4cf79d15947 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] Created volume successfully. >2018-08-23 07:37:34.849 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d00d1445-f4ca-4619-b101-41802a06ff8d 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] initialize_connection: Did not find a preferred node for volume volume-244c589e-3a0c-431a-b8f4-39c356a7bcc4. >2018-08-23 07:37:35.438 505523 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 07:37:52.292 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d00d1445-f4ca-4619-b101-41802a06ff8d 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 07:37:52.612 505523 INFO cinder.volume.manager [req-d00d1445-f4ca-4619-b101-41802a06ff8d 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] Copy volume to image completed successfully. >2018-08-23 07:37:56.214 505523 INFO cinder.volume.manager [req-75312463-7869-4b27-ac32-dafc00ea5a14 5cae1207124941189232f66b360af43e 62bec49c4766440388cbf2e1d6a9051b - default default] Deleted volume successfully. >2018-08-23 07:38:02.296 505523 INFO cinder.volume.flows.manager.create_volume [req-7813d043-e9b1-488b-bcf5-f6f576c48658 361af28460da46ad941610e0fa4247c6 5869800344a24b2784314eacf5d1634b - default default] Volume aa6f6b41-f5fc-4cbe-a934-d12a4fafe426: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-aa6f6b41-f5fc-4cbe-a934-d12a4fafe426'} >2018-08-23 07:38:03.094 505523 INFO cinder.volume.flows.manager.create_volume [req-7813d043-e9b1-488b-bcf5-f6f576c48658 361af28460da46ad941610e0fa4247c6 5869800344a24b2784314eacf5d1634b - default default] Volume volume-aa6f6b41-f5fc-4cbe-a934-d12a4fafe426 (aa6f6b41-f5fc-4cbe-a934-d12a4fafe426): created successfully >2018-08-23 07:38:03.098 505523 INFO cinder.volume.manager [req-7813d043-e9b1-488b-bcf5-f6f576c48658 361af28460da46ad941610e0fa4247c6 5869800344a24b2784314eacf5d1634b - default default] Created volume successfully. >2018-08-23 07:43:08.999 505523 INFO cinder.volume.manager [req-a5f2104d-874b-4dbd-9e28-e5ab188e322c 361af28460da46ad941610e0fa4247c6 5869800344a24b2784314eacf5d1634b - default default] Deleted volume successfully. >2018-08-23 07:43:18.220 505523 INFO cinder.volume.flows.manager.create_volume [req-df4af5b4-e8ed-45e9-9af4-86633525b7c2 e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] Volume d30969d1-04d0-453d-bee5-96ba776552c5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d30969d1-04d0-453d-bee5-96ba776552c5'} >2018-08-23 07:43:18.798 505523 INFO cinder.volume.flows.manager.create_volume [req-df4af5b4-e8ed-45e9-9af4-86633525b7c2 e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] Volume volume-d30969d1-04d0-453d-bee5-96ba776552c5 (d30969d1-04d0-453d-bee5-96ba776552c5): created successfully >2018-08-23 07:43:18.802 505523 INFO cinder.volume.manager [req-df4af5b4-e8ed-45e9-9af4-86633525b7c2 e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] Created volume successfully. >2018-08-23 07:43:21.824 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-303ae9b1-c363-4ea8-bbe1-5bd23039dfcb e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] Unable to get remote copy information for volume volume-d30969d1-04d0-453d-bee5-96ba776552c5 >2018-08-23 07:43:22.104 505523 INFO cinder.volume.manager [req-303ae9b1-c363-4ea8-bbe1-5bd23039dfcb e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] Extend volume completed successfully. >2018-08-23 07:43:25.126 505523 INFO cinder.volume.manager [req-00bd00e3-3fe9-49f9-9262-82d6ca22ec3b e6c8521140c54bcdadbd185fd8492b8b 3d87f742216c4906b4008464ba4c2f67 - default default] Deleted volume successfully. >2018-08-23 07:43:31.219 505523 INFO cinder.volume.flows.manager.create_volume [req-f079cfa0-5004-4e7f-ae2c-c1f1b7881255 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume f288e65e-9729-4a0c-9541-da3c133c625b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f288e65e-9729-4a0c-9541-da3c133c625b'} >2018-08-23 07:43:31.742 505523 INFO cinder.volume.flows.manager.create_volume [req-f079cfa0-5004-4e7f-ae2c-c1f1b7881255 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume volume-f288e65e-9729-4a0c-9541-da3c133c625b (f288e65e-9729-4a0c-9541-da3c133c625b): created successfully >2018-08-23 07:43:31.746 505523 INFO cinder.volume.manager [req-f079cfa0-5004-4e7f-ae2c-c1f1b7881255 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Created volume successfully. >2018-08-23 07:43:34.858 505523 INFO cinder.volume.flows.manager.create_volume [req-f1a37098-6f66-40fa-a34e-9ee387e4601e 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume d4fff43a-b094-427d-bef3-2878349f656f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d4fff43a-b094-427d-bef3-2878349f656f'} >2018-08-23 07:43:35.381 505523 INFO cinder.volume.flows.manager.create_volume [req-f1a37098-6f66-40fa-a34e-9ee387e4601e 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume volume-d4fff43a-b094-427d-bef3-2878349f656f (d4fff43a-b094-427d-bef3-2878349f656f): created successfully >2018-08-23 07:43:35.385 505523 INFO cinder.volume.manager [req-f1a37098-6f66-40fa-a34e-9ee387e4601e 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Created volume successfully. >2018-08-23 07:43:36.932 505523 INFO cinder.volume.manager [req-6d78c725-fa16-4a2c-87fb-3f8e507d7c34 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Deleted volume successfully. >2018-08-23 07:43:38.327 505523 INFO cinder.volume.manager [req-5be573d7-dce1-4d17-aaa5-bebcbfe76cab 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Deleted volume successfully. >2018-08-23 07:43:38.868 505523 INFO cinder.volume.flows.manager.create_volume [req-ba5b5a50-da7b-424c-804b-9a54db995c00 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume 651af339-e83e-4a72-bc9b-60d1c282bb46: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-651af339-e83e-4a72-bc9b-60d1c282bb46'} >2018-08-23 07:43:39.390 505523 INFO cinder.volume.flows.manager.create_volume [req-ba5b5a50-da7b-424c-804b-9a54db995c00 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume volume-651af339-e83e-4a72-bc9b-60d1c282bb46 (651af339-e83e-4a72-bc9b-60d1c282bb46): created successfully >2018-08-23 07:43:39.393 505523 INFO cinder.volume.manager [req-ba5b5a50-da7b-424c-804b-9a54db995c00 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Created volume successfully. >2018-08-23 07:43:40.270 505523 INFO cinder.volume.flows.manager.create_volume [req-db675545-d7b4-49be-b203-a60d6b292a98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume 536d8999-836c-426f-85a0-552e9306f870: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-536d8999-836c-426f-85a0-552e9306f870', 'source_volid': '651af339-e83e-4a72-bc9b-60d1c282bb46', 'source_volstatus': u'available'} >2018-08-23 07:43:41.730 505523 INFO cinder.volume.flows.manager.create_volume [req-db675545-d7b4-49be-b203-a60d6b292a98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume volume-536d8999-836c-426f-85a0-552e9306f870 (536d8999-836c-426f-85a0-552e9306f870): created successfully >2018-08-23 07:43:41.734 505523 INFO cinder.volume.manager [req-db675545-d7b4-49be-b203-a60d6b292a98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Created volume successfully. >2018-08-23 07:43:43.564 505523 INFO cinder.volume.flows.manager.create_volume [req-4baf4d03-3c06-4bc6-87d0-d2d573b51272 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume 835e0be6-318f-41c3-8567-25c42dfed31b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-835e0be6-318f-41c3-8567-25c42dfed31b'} >2018-08-23 07:43:44.148 505523 INFO cinder.volume.flows.manager.create_volume [req-4baf4d03-3c06-4bc6-87d0-d2d573b51272 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume volume-835e0be6-318f-41c3-8567-25c42dfed31b (835e0be6-318f-41c3-8567-25c42dfed31b): created successfully >2018-08-23 07:43:44.152 505523 INFO cinder.volume.manager [req-4baf4d03-3c06-4bc6-87d0-d2d573b51272 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Created volume successfully. >2018-08-23 07:43:46.038 505523 INFO cinder.volume.manager [req-8eb29687-d52d-4b2d-9922-748974a4c830 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Deleted volume successfully. >2018-08-23 07:43:50.470 505523 INFO cinder.volume.manager [req-fe5ef698-25e7-46dc-b4e4-5ea909b76d6f 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Deleted volume successfully. >2018-08-23 07:43:51.872 505523 INFO cinder.volume.flows.manager.create_volume [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume 7e8373a9-1f4c-4d0b-88d5-9d0c14824f69: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-7e8373a9-1f4c-4d0b-88d5-9d0c14824f69', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f269573b610>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 07:43:53.186 505523 INFO cinder.image.image_utils [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Image download 12.00 MB at 9.15 MB/s >2018-08-23 07:44:02.358 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] initialize_connection: Did not find a preferred node for volume volume-7e8373a9-1f4c-4d0b-88d5-9d0c14824f69. >2018-08-23 07:44:03.031 505523 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 07:44:07.715 505523 INFO cinder.image.image_utils [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 07:44:09.571 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 07:44:09.915 505523 INFO cinder.volume.flows.manager.create_volume [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume volume-7e8373a9-1f4c-4d0b-88d5-9d0c14824f69 (7e8373a9-1f4c-4d0b-88d5-9d0c14824f69): created successfully >2018-08-23 07:44:09.919 505523 INFO cinder.volume.manager [req-f42a44c9-16c6-49ab-8938-73f240ec1e98 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Created volume successfully. >2018-08-23 07:44:10.991 505523 INFO cinder.volume.flows.manager.create_volume [req-6058a646-ba60-4b72-a0d0-f5bb424b4c79 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume 57d21d38-eed3-4db0-9162-e5114de6e817: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-57d21d38-eed3-4db0-9162-e5114de6e817'} >2018-08-23 07:44:11.514 505523 INFO cinder.volume.flows.manager.create_volume [req-6058a646-ba60-4b72-a0d0-f5bb424b4c79 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Volume volume-57d21d38-eed3-4db0-9162-e5114de6e817 (57d21d38-eed3-4db0-9162-e5114de6e817): created successfully >2018-08-23 07:44:11.518 505523 INFO cinder.volume.manager [req-6058a646-ba60-4b72-a0d0-f5bb424b4c79 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Created volume successfully. >2018-08-23 07:44:13.333 505523 INFO cinder.volume.manager [req-8e8c0143-8823-4b30-91b2-72f2dc300c3f 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Deleted volume successfully. >2018-08-23 07:44:14.567 505523 INFO cinder.volume.manager [req-2e265167-b89a-467a-989f-46074d5d9a23 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Deleted volume successfully. >2018-08-23 07:44:16.026 505523 INFO cinder.volume.manager [req-c5e8cdcc-26f3-4c70-b3fd-c8bbdd78bad5 152119624fcf4729a7e16ee9dbc8019f c30aab9c00e242789911e9b521e17c88 - default default] Deleted volume successfully. >2018-08-23 07:44:25.179 505523 INFO cinder.volume.flows.manager.create_volume [req-65b1edbb-cdca-4a85-a8d9-5fadde6ade3a e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Volume 317ed324-012f-4929-81fb-fbbc246a448b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-317ed324-012f-4929-81fb-fbbc246a448b'} >2018-08-23 07:44:25.704 505523 INFO cinder.volume.flows.manager.create_volume [req-65b1edbb-cdca-4a85-a8d9-5fadde6ade3a e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Volume volume-317ed324-012f-4929-81fb-fbbc246a448b (317ed324-012f-4929-81fb-fbbc246a448b): created successfully >2018-08-23 07:44:25.708 505523 INFO cinder.volume.manager [req-65b1edbb-cdca-4a85-a8d9-5fadde6ade3a e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Created volume successfully. >2018-08-23 07:44:27.022 505523 INFO cinder.volume.flows.manager.create_volume [req-e5c5179c-3fd3-445d-846c-6f5cfccae5bd e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Volume 4d761931-d585-4794-8260-21da5064fc15: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4d761931-d585-4794-8260-21da5064fc15'} >2018-08-23 07:44:27.565 505523 INFO cinder.volume.flows.manager.create_volume [req-e5c5179c-3fd3-445d-846c-6f5cfccae5bd e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Volume volume-4d761931-d585-4794-8260-21da5064fc15 (4d761931-d585-4794-8260-21da5064fc15): created successfully >2018-08-23 07:44:27.569 505523 INFO cinder.volume.manager [req-e5c5179c-3fd3-445d-846c-6f5cfccae5bd e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Created volume successfully. >2018-08-23 07:44:29.022 505464 INFO cinder.volume.flows.manager.create_volume [req-8f8de0e1-c5a0-492b-af11-b267a2cb4c1f e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Volume 577c78f6-4fca-422a-aa4d-68091526c489: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-577c78f6-4fca-422a-aa4d-68091526c489'} >2018-08-23 07:44:32.550 505464 INFO cinder.volume.flows.manager.create_volume [req-8f8de0e1-c5a0-492b-af11-b267a2cb4c1f e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Volume volume-577c78f6-4fca-422a-aa4d-68091526c489 (577c78f6-4fca-422a-aa4d-68091526c489): created successfully >2018-08-23 07:44:32.553 505464 INFO cinder.volume.manager [req-8f8de0e1-c5a0-492b-af11-b267a2cb4c1f e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Created volume successfully. >2018-08-23 07:44:38.879 505523 INFO cinder.volume.manager [req-084cb208-b095-4c6d-9373-dff8b1ea3b71 e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Deleted volume successfully. >2018-08-23 07:44:38.899 505464 INFO cinder.volume.manager [req-390571db-2f2b-48c7-b15a-c226c6c4ce2d e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Deleted volume successfully. >2018-08-23 07:44:39.120 505523 INFO cinder.volume.manager [req-8f28dd35-b4b3-495f-84ff-518e45c69ef1 e84a31098cd34bb7823608aa58cd7901 c68d479b2c2a4dfcb95cfd17ec0b4747 - default default] Deleted volume successfully. >2018-08-23 07:44:44.372 505523 INFO cinder.volume.flows.manager.create_volume [req-b9399c3d-95f3-4f2d-a977-382560e12414 3f5240245eb7482498c0860fa446e8a3 a067d8c84f1145cfb2145f2c772b2116 - default default] Volume 90beccd0-07eb-4065-b85b-39794fbea3fa: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-90beccd0-07eb-4065-b85b-39794fbea3fa'} >2018-08-23 07:44:44.907 505523 INFO cinder.volume.flows.manager.create_volume [req-b9399c3d-95f3-4f2d-a977-382560e12414 3f5240245eb7482498c0860fa446e8a3 a067d8c84f1145cfb2145f2c772b2116 - default default] Volume volume-90beccd0-07eb-4065-b85b-39794fbea3fa (90beccd0-07eb-4065-b85b-39794fbea3fa): created successfully >2018-08-23 07:44:44.911 505523 INFO cinder.volume.manager [req-b9399c3d-95f3-4f2d-a977-382560e12414 3f5240245eb7482498c0860fa446e8a3 a067d8c84f1145cfb2145f2c772b2116 - default default] Created volume successfully. >2018-08-23 07:50:04.948 505523 INFO cinder.volume.manager [req-c92b7394-68dd-4b4d-bf92-143bc8f2a156 3f5240245eb7482498c0860fa446e8a3 a067d8c84f1145cfb2145f2c772b2116 - default default] Deleted volume successfully. >2018-08-23 07:50:10.367 505523 INFO cinder.volume.flows.manager.create_volume [req-8d83d2d7-2b3a-4302-a167-ada7b953c3b6 b63a8183c0d04aeda4c2ab8261fb10d2 f96c6a7f82034523be53f58719e972da - default default] Volume dcf15e14-5a84-46f9-8876-bd264494c4aa: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-dcf15e14-5a84-46f9-8876-bd264494c4aa'} >2018-08-23 07:50:10.893 505523 INFO cinder.volume.flows.manager.create_volume [req-8d83d2d7-2b3a-4302-a167-ada7b953c3b6 b63a8183c0d04aeda4c2ab8261fb10d2 f96c6a7f82034523be53f58719e972da - default default] Volume volume-dcf15e14-5a84-46f9-8876-bd264494c4aa (dcf15e14-5a84-46f9-8876-bd264494c4aa): created successfully >2018-08-23 07:50:10.897 505523 INFO cinder.volume.manager [req-8d83d2d7-2b3a-4302-a167-ada7b953c3b6 b63a8183c0d04aeda4c2ab8261fb10d2 f96c6a7f82034523be53f58719e972da - default default] Created volume successfully. >2018-08-23 07:50:13.493 505523 INFO cinder.volume.manager [req-88cc0027-ac09-4b59-927d-9f425ad55682 b63a8183c0d04aeda4c2ab8261fb10d2 f96c6a7f82034523be53f58719e972da - default default] Deleted volume successfully. >2018-08-23 07:51:17.638 505523 INFO cinder.volume.flows.manager.create_volume [req-db915b51-2aea-41ac-b010-e18a026306f2 af12a1440c9a48d7bb49f29ea8fea123 c22b3d4f5d9f4e7eb63202297223821b - default default] Volume 0323fcf5-9571-4efc-9e9e-cb44d82b50e6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0323fcf5-9571-4efc-9e9e-cb44d82b50e6'} >2018-08-23 07:51:18.231 505523 INFO cinder.volume.flows.manager.create_volume [req-db915b51-2aea-41ac-b010-e18a026306f2 af12a1440c9a48d7bb49f29ea8fea123 c22b3d4f5d9f4e7eb63202297223821b - default default] Volume volume-0323fcf5-9571-4efc-9e9e-cb44d82b50e6 (0323fcf5-9571-4efc-9e9e-cb44d82b50e6): created successfully >2018-08-23 07:51:18.235 505523 INFO cinder.volume.manager [req-db915b51-2aea-41ac-b010-e18a026306f2 af12a1440c9a48d7bb49f29ea8fea123 c22b3d4f5d9f4e7eb63202297223821b - default default] Created volume successfully. >2018-08-23 07:51:21.649 505523 INFO cinder.volume.manager [req-47907ebd-fc2d-4c59-804f-b1f92717519e af12a1440c9a48d7bb49f29ea8fea123 c22b3d4f5d9f4e7eb63202297223821b - default default] Deleted volume successfully. >2018-08-23 07:51:28.319 505523 INFO cinder.volume.flows.manager.create_volume [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Volume d351febb-46b7-4a83-8393-e88e774d046c: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-d351febb-46b7-4a83-8393-e88e774d046c', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f2695707150>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 07:51:29.133 505523 INFO cinder.image.image_utils [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-23 07:51:37.669 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] initialize_connection: Did not find a preferred node for volume volume-d351febb-46b7-4a83-8393-e88e774d046c. >2018-08-23 07:51:38.253 505523 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 07:51:43.030 505523 INFO cinder.image.image_utils [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 07:51:44.971 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 07:51:45.322 505523 INFO cinder.volume.flows.manager.create_volume [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Volume volume-d351febb-46b7-4a83-8393-e88e774d046c (d351febb-46b7-4a83-8393-e88e774d046c): created successfully >2018-08-23 07:51:45.326 505523 INFO cinder.volume.manager [req-1702001a-e2a0-480c-802b-988f7786a1f5 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Created volume successfully. >2018-08-23 07:51:46.689 505523 INFO cinder.volume.flows.manager.create_volume [req-3dd80efb-b41b-45c0-99ef-17259bd32dba 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Volume dbf96139-aa9b-42d8-80e6-b7821859d4b3: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-dbf96139-aa9b-42d8-80e6-b7821859d4b3', 'source_volid': 'd351febb-46b7-4a83-8393-e88e774d046c', 'source_volstatus': u'available'} >2018-08-23 07:51:48.166 505523 INFO cinder.volume.flows.manager.create_volume [req-3dd80efb-b41b-45c0-99ef-17259bd32dba 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Volume volume-dbf96139-aa9b-42d8-80e6-b7821859d4b3 (dbf96139-aa9b-42d8-80e6-b7821859d4b3): created successfully >2018-08-23 07:51:48.170 505523 INFO cinder.volume.manager [req-3dd80efb-b41b-45c0-99ef-17259bd32dba 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Created volume successfully. >2018-08-23 07:51:52.443 505523 INFO cinder.volume.manager [req-7730f802-85fc-4b2a-ae5a-55d0fa727d26 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Deleted volume successfully. >2018-08-23 07:51:53.391 505523 INFO cinder.volume.manager [req-e9a1f81f-c8de-46c6-b391-e7144ed7db0e 069750f76b694a719ae3267c50f90367 7c7a5db99d0447ecb0c9910bde08850c - default default] Deleted volume successfully. >2018-08-23 07:51:58.725 505523 INFO cinder.volume.flows.manager.create_volume [req-290ddcc1-90cb-4060-b92a-2e0a8e1638f9 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Volume 262042e0-8b84-400e-bb65-936be656d961: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-262042e0-8b84-400e-bb65-936be656d961'} >2018-08-23 07:51:59.578 505523 INFO cinder.volume.flows.manager.create_volume [req-290ddcc1-90cb-4060-b92a-2e0a8e1638f9 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Volume volume-262042e0-8b84-400e-bb65-936be656d961 (262042e0-8b84-400e-bb65-936be656d961): created successfully >2018-08-23 07:51:59.582 505523 INFO cinder.volume.manager [req-290ddcc1-90cb-4060-b92a-2e0a8e1638f9 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Created volume successfully. >2018-08-23 07:52:00.830 505464 INFO cinder.volume.flows.manager.create_volume [req-41a6e46d-acb4-47c3-869b-a5980556e655 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Volume 96ec75d1-03d0-4824-82bf-657617de9103: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-96ec75d1-03d0-4824-82bf-657617de9103'} >2018-08-23 07:52:01.725 505464 INFO cinder.volume.flows.manager.create_volume [req-41a6e46d-acb4-47c3-869b-a5980556e655 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Volume volume-96ec75d1-03d0-4824-82bf-657617de9103 (96ec75d1-03d0-4824-82bf-657617de9103): created successfully >2018-08-23 07:52:01.729 505464 INFO cinder.volume.manager [req-41a6e46d-acb4-47c3-869b-a5980556e655 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Created volume successfully. >2018-08-23 07:52:02.638 505523 INFO cinder.volume.flows.manager.create_volume [req-e37907c2-85a3-457c-8e0b-209cd257148c 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Volume 837f7ddb-bc2b-4382-9c82-706a3680baec: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-837f7ddb-bc2b-4382-9c82-706a3680baec'} >2018-08-23 07:52:03.511 505523 INFO cinder.volume.flows.manager.create_volume [req-e37907c2-85a3-457c-8e0b-209cd257148c 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Volume volume-837f7ddb-bc2b-4382-9c82-706a3680baec (837f7ddb-bc2b-4382-9c82-706a3680baec): created successfully >2018-08-23 07:52:03.515 505523 INFO cinder.volume.manager [req-e37907c2-85a3-457c-8e0b-209cd257148c 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Created volume successfully. >2018-08-23 07:52:07.300 505523 INFO cinder.volume.manager [req-61ed4861-f89f-46b7-96c2-9fce07ff228f 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Deleted volume successfully. >2018-08-23 07:52:07.593 505464 INFO cinder.volume.manager [req-e7c66915-349b-4850-ad68-26f48a7f4191 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Deleted volume successfully. >2018-08-23 07:52:07.947 505523 INFO cinder.volume.manager [req-2d0e2d02-4cd0-4dbd-b749-c2560ece47ef 17a53df7d8874e65abbcbebab119edb0 f83ae341700249f297286d64647e3f41 - default default] Deleted volume successfully. >2018-08-23 07:52:29.072 505464 INFO cinder.volume.flows.manager.create_volume [req-f392e4e3-f66f-406d-980a-15811eed8c03 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Volume a3b2bc23-a30e-48c2-a3d8-eed38762e406: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406'} >2018-08-23 07:52:30.074 505464 INFO cinder.volume.flows.manager.create_volume [req-f392e4e3-f66f-406d-980a-15811eed8c03 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Volume volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 (a3b2bc23-a30e-48c2-a3d8-eed38762e406): created successfully >2018-08-23 07:52:30.078 505464 INFO cinder.volume.manager [req-f392e4e3-f66f-406d-980a-15811eed8c03 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Created volume successfully. >2018-08-23 07:52:31.284 505464 INFO cinder.rpc [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 07:52:31.289 505464 INFO cinder.rpc [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 07:52:31.436 505523 INFO cinder.volume.flows.manager.create_volume [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Volume 94a94bf9-09a3-4068-956a-df77868ccdc5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-94a94bf9-09a3-4068-956a-df77868ccdc5'} >2018-08-23 07:52:32.354 505523 INFO cinder.volume.flows.manager.create_volume [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Volume volume-94a94bf9-09a3-4068-956a-df77868ccdc5 (94a94bf9-09a3-4068-956a-df77868ccdc5): created successfully >2018-08-23 07:52:32.358 505523 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Created volume successfully. >2018-08-23 07:52:32.427 505464 INFO oslo.privsep.daemon [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpwEYbCG/privsep.sock'] >2018-08-23 07:52:33.201 505464 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-23 07:52:33.299 505464 INFO oslo.privsep.daemon [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Spawned new privsep daemon via rootwrap >2018-08-23 07:52:33.243 119581 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-23 07:52:33.247 119581 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-23 07:52:33.249 119581 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-23 07:52:33.249 119581 INFO oslo.privsep.daemon [-] privsep daemon running as pid 119581 >2018-08-23 07:52:48.609 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] initialize_connection: Did not find a preferred node for volume volume-94a94bf9-09a3-4068-956a-df77868ccdc5. >2018-08-23 07:52:48.984 505523 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Initialize volume connection completed successfully. >2018-08-23 07:52:49.375 505464 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 07:52:52.562 505464 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] initialize_connection: Did not find a preferred node for volume volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406. >2018-08-23 07:52:52.778 505464 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Initialize volume connection completed successfully. >2018-08-23 07:52:53.163 505464 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 07:52:58.529 505464 INFO cinder.volume.utils [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Volume copy 1024.00 MB at 427.28 MB/s >2018-08-23 07:53:03.852 505523 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Terminate volume connection completed successfully. >2018-08-23 07:53:03.894 505523 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Remove volume export completed successfully. >2018-08-23 07:53:06.062 505464 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 07:53:06.339 505464 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Terminate volume connection completed successfully. >2018-08-23 07:53:06.374 505464 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Remove volume export completed successfully. >2018-08-23 07:53:06.644 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 07:53:11.576 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Error running SSH command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 07:53:11.577 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 07:53:11.578 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Error running SSH command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 07:53:11.579 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406', '"volume-94a94bf9-09a3-4068-956a-df77868ccdc5"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406 "volume-94a94bf9-09a3-4068-956a-df77868ccdc5" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 07:53:11.580 505523 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Unable to rename the logical volume for volume: a3b2bc23-a30e-48c2-a3d8-eed38762e406: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-a3b2bc23-a30e-48c2-a3d8-eed38762e406', '"volume-94a94bf9-09a3-4068-956a-df77868ccdc5"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-23 07:53:11.750 505464 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Complete-Migrate volume completed successfully. >2018-08-23 07:53:11.751 505464 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Migrate volume completed successfully. >2018-08-23 07:53:12.219 505464 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Retype volume completed successfully. >2018-08-23 07:53:12.858 505464 INFO cinder.volume.manager [req-a714b17f-0598-49e5-86d2-b54c1fdbb6e9 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Deleted volume successfully. >2018-08-23 07:53:15.312 505523 WARNING cinder.quota [req-5577d1e3-569a-470b-9a93-bc60bf370026 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1518157014 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1518157014, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:53:15.313 505523 WARNING cinder.quota [req-5577d1e3-569a-470b-9a93-bc60bf370026 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1518157014 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1518157014, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 07:53:15.564 505523 INFO cinder.volume.manager [req-5577d1e3-569a-470b-9a93-bc60bf370026 ca39de1e3d404b3ea8b62335ea86abe7 a3196e6772c8486fa8f8ddc604613160 - default default] Deleted volume successfully. >2018-08-23 07:53:36.496 505523 INFO cinder.volume.flows.manager.create_volume [req-cd2690fc-bd43-46e4-8130-4523fd5e94c0 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Volume 93cc908b-a81a-499e-809e-4a2134e0d612: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-93cc908b-a81a-499e-809e-4a2134e0d612'} >2018-08-23 07:53:37.037 505523 INFO cinder.volume.flows.manager.create_volume [req-cd2690fc-bd43-46e4-8130-4523fd5e94c0 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Volume volume-93cc908b-a81a-499e-809e-4a2134e0d612 (93cc908b-a81a-499e-809e-4a2134e0d612): created successfully >2018-08-23 07:53:37.041 505523 INFO cinder.volume.manager [req-cd2690fc-bd43-46e4-8130-4523fd5e94c0 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Created volume successfully. >2018-08-23 07:53:45.218 505523 INFO cinder.volume.manager [req-fafc1fdb-ce0f-4e5c-8008-ee89c5c44a5b 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Create snapshot completed successfully >2018-08-23 07:53:53.460 505523 INFO cinder.volume.manager [req-0e5e7474-1d41-4c55-abeb-e771d6d596f4 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Create snapshot completed successfully >2018-08-23 07:53:56.570 505523 INFO cinder.volume.manager [req-7d359906-9a5a-49b2-99ad-959e818d3d5b dbde7e5ca2be4588ae12e8aea9c7e1b2 a5e170decd3d4e97b2298e6be31dfae6 - default default] Delete snapshot completed successfully. >2018-08-23 07:54:04.563 505523 INFO cinder.volume.manager [req-77c55785-8b41-48d4-a150-a593cca34d00 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Create snapshot completed successfully >2018-08-23 07:54:06.915 505523 INFO cinder.volume.manager [req-c010d202-1c3b-4812-b1cd-d1a7c2a328bf dbde7e5ca2be4588ae12e8aea9c7e1b2 a5e170decd3d4e97b2298e6be31dfae6 - default default] Delete snapshot completed successfully. >2018-08-23 07:54:14.682 505523 INFO cinder.volume.manager [req-32e175e6-c558-4026-be22-b7f2c64f06e1 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Create snapshot completed successfully >2018-08-23 07:54:17.182 505523 INFO cinder.volume.manager [req-2605e04c-121a-4ef8-8e86-73d4ff296d7d dbde7e5ca2be4588ae12e8aea9c7e1b2 a5e170decd3d4e97b2298e6be31dfae6 - default default] Delete snapshot completed successfully. >2018-08-23 07:54:24.489 505523 INFO cinder.volume.manager [req-fd2a1502-5c60-49e0-bf5b-6ead7e5e84da 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Create snapshot completed successfully >2018-08-23 07:54:27.706 505523 INFO cinder.volume.manager [req-c44f9462-858d-4957-b886-d2877c01b416 dbde7e5ca2be4588ae12e8aea9c7e1b2 a5e170decd3d4e97b2298e6be31dfae6 - default default] Delete snapshot completed successfully. >2018-08-23 07:54:30.054 505523 INFO cinder.volume.manager [req-abd4bc9d-553e-4b3a-bc8a-d4a8389970c7 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Delete snapshot completed successfully. >2018-08-23 07:54:31.852 505523 INFO cinder.volume.manager [req-ba7f6916-07ba-4e97-b00a-c137a9a896d7 5f7e6a6790a749f894a811bdb87213ff d2bd9ba5d49f408fa5b2927c851e4345 - default default] Deleted volume successfully. >2018-08-23 07:54:36.703 505523 INFO cinder.volume.flows.manager.create_volume [req-d737e57e-bb92-4d09-bb88-a1880f71265d 3b7df3b0172540349f1f4cbd5cf1b8ed 49b2bbf4caf44c4b9935e3e80c079e13 - default default] Volume 99a2beb1-1401-4534-be28-8973611140c0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a2beb1-1401-4534-be28-8973611140c0'} >2018-08-23 07:54:37.238 505523 INFO cinder.volume.flows.manager.create_volume [req-d737e57e-bb92-4d09-bb88-a1880f71265d 3b7df3b0172540349f1f4cbd5cf1b8ed 49b2bbf4caf44c4b9935e3e80c079e13 - default default] Volume volume-99a2beb1-1401-4534-be28-8973611140c0 (99a2beb1-1401-4534-be28-8973611140c0): created successfully >2018-08-23 07:54:37.242 505523 INFO cinder.volume.manager [req-d737e57e-bb92-4d09-bb88-a1880f71265d 3b7df3b0172540349f1f4cbd5cf1b8ed 49b2bbf4caf44c4b9935e3e80c079e13 - default default] Created volume successfully. >2018-08-23 07:54:45.565 505523 INFO cinder.volume.manager [req-904323b7-05a0-42bf-b87f-7a738f054f83 3b7df3b0172540349f1f4cbd5cf1b8ed 49b2bbf4caf44c4b9935e3e80c079e13 - default default] Create snapshot completed successfully >2018-08-23 07:54:49.028 505523 INFO cinder.volume.manager [req-7820a682-4a37-406c-963b-bce68c9215a4 3b7df3b0172540349f1f4cbd5cf1b8ed 49b2bbf4caf44c4b9935e3e80c079e13 - default default] Delete snapshot completed successfully. >2018-08-23 07:54:50.521 505523 INFO cinder.volume.manager [req-4182d4be-ea89-42c0-ba60-8e8f482f892d 3b7df3b0172540349f1f4cbd5cf1b8ed 49b2bbf4caf44c4b9935e3e80c079e13 - default default] Deleted volume successfully. >2018-08-23 07:54:54.667 505523 INFO cinder.volume.flows.manager.create_volume [req-ac2f2998-bf4d-4219-8424-e0d80999b648 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Volume fb5f926c-3519-4c92-bb8a-34907cf57b40: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fb5f926c-3519-4c92-bb8a-34907cf57b40'} >2018-08-23 07:54:55.219 505523 INFO cinder.volume.flows.manager.create_volume [req-ac2f2998-bf4d-4219-8424-e0d80999b648 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Volume volume-fb5f926c-3519-4c92-bb8a-34907cf57b40 (fb5f926c-3519-4c92-bb8a-34907cf57b40): created successfully >2018-08-23 07:54:55.223 505523 INFO cinder.volume.manager [req-ac2f2998-bf4d-4219-8424-e0d80999b648 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Created volume successfully. >2018-08-23 07:55:03.201 505523 INFO cinder.volume.manager [req-24bc71d9-ea73-4d4f-a422-a99c80bab968 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Create snapshot completed successfully >2018-08-23 07:55:21.303 505523 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b919011f-3733-41c2-a1df-8c552d66f8c5 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] initialize_connection: Did not find a preferred node for volume volume-365b8365-b3e4-4d61-b3f7-ffb945912cff. >2018-08-23 07:55:21.505 505523 INFO cinder.volume.manager [req-b919011f-3733-41c2-a1df-8c552d66f8c5 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Initialize volume connection completed successfully. >2018-08-23 07:55:44.044 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b919011f-3733-41c2-a1df-8c552d66f8c5 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 07:55:44.366 505523 INFO cinder.volume.manager [req-b919011f-3733-41c2-a1df-8c552d66f8c5 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Terminate volume connection completed successfully. >2018-08-23 07:55:44.410 505523 INFO cinder.volume.manager [req-b919011f-3733-41c2-a1df-8c552d66f8c5 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Remove volume export completed successfully. >2018-08-23 07:55:45.689 505523 INFO cinder.volume.manager [req-b919011f-3733-41c2-a1df-8c552d66f8c5 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Deleted volume successfully. >2018-08-23 08:00:52.080 505523 INFO cinder.volume.manager [req-3389d7a2-ad42-4024-870b-5b88646bef1b 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Create snapshot completed successfully >2018-08-23 08:00:56.446 505523 INFO cinder.volume.manager [req-35b541bd-f1fe-4565-9549-3392ed49cea9 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Delete snapshot completed successfully. >2018-08-23 08:01:02.192 505523 INFO cinder.volume.manager [req-a7de09b6-2691-49a9-88ac-8fb7bce7f93e 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Create snapshot completed successfully >2018-08-23 08:06:05.259 505523 INFO cinder.volume.flows.manager.create_volume [req-00bf2fba-b597-49e5-9a0c-e2706f663e99 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Volume f8a27ecb-a1d4-4eca-8322-81bf7e1ef473: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f8a27ecb-a1d4-4eca-8322-81bf7e1ef473'} >2018-08-23 08:06:05.788 505523 INFO cinder.volume.flows.manager.create_volume [req-00bf2fba-b597-49e5-9a0c-e2706f663e99 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Volume volume-f8a27ecb-a1d4-4eca-8322-81bf7e1ef473 (f8a27ecb-a1d4-4eca-8322-81bf7e1ef473): created successfully >2018-08-23 08:06:05.791 505523 INFO cinder.volume.manager [req-00bf2fba-b597-49e5-9a0c-e2706f663e99 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Created volume successfully. >2018-08-23 08:06:08.895 505523 INFO cinder.volume.manager [req-cf0182a7-30f0-4a16-8c1d-fad9e897b06f 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Create snapshot completed successfully >2018-08-23 08:06:10.364 505523 INFO cinder.volume.flows.manager.create_volume [req-d255981b-fab8-4901-8f89-8c4ce185d4b2 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Volume 3e23718a-ba2f-4371-99f8-3dca6c547012: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': 'f303db90-f51b-43ed-a438-f1f8f3d3783d', 'volume_name': 'volume-3e23718a-ba2f-4371-99f8-3dca6c547012'} >2018-08-23 08:06:32.409 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-d255981b-fab8-4901-8f89-8c4ce185d4b2 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Unable to get remote copy information for volume volume-3e23718a-ba2f-4371-99f8-3dca6c547012 >2018-08-23 08:06:32.712 505523 INFO cinder.volume.flows.manager.create_volume [req-d255981b-fab8-4901-8f89-8c4ce185d4b2 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Volume volume-3e23718a-ba2f-4371-99f8-3dca6c547012 (3e23718a-ba2f-4371-99f8-3dca6c547012): created successfully >2018-08-23 08:06:32.716 505523 INFO cinder.volume.manager [req-d255981b-fab8-4901-8f89-8c4ce185d4b2 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Created volume successfully. >2018-08-23 08:06:34.364 505523 INFO cinder.volume.manager [req-5ace941a-53d9-43bb-b691-441c3983872c 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Deleted volume successfully. >2018-08-23 08:06:34.811 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-8737b7a5-527c-4634-a73d-2233af49fbd7 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Tried to delete non-existent vdisk snapshot-b66bf5db-aad8-4889-918a-07aa0ddef231. >2018-08-23 08:06:34.949 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-ab2f5945-fccb-4f41-8793-79c23b968019 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Tried to delete non-existent vdisk snapshot-f2a6df46-e4b0-4023-bc11-861ee010d462. >2018-08-23 08:06:35.028 505523 INFO cinder.volume.manager [req-8737b7a5-527c-4634-a73d-2233af49fbd7 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Delete snapshot completed successfully. >2018-08-23 08:06:35.087 505523 INFO cinder.volume.manager [req-ab2f5945-fccb-4f41-8793-79c23b968019 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Delete snapshot completed successfully. >2018-08-23 08:06:37.216 505523 INFO cinder.volume.manager [req-6ac46ffa-91de-499a-93c0-2f0f15a5dd8a 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Delete snapshot completed successfully. >2018-08-23 08:06:38.509 505523 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d035f6a-d577-4b1d-9fee-d9f20a7ac233 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Tried to delete non-existent vdisk volume-fb5f926c-3519-4c92-bb8a-34907cf57b40. >2018-08-23 08:06:38.902 505523 INFO cinder.volume.manager [req-0d035f6a-d577-4b1d-9fee-d9f20a7ac233 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Deleted volume successfully. >2018-08-23 08:06:39.423 505464 INFO cinder.volume.flows.manager.create_volume [req-41ead9b8-017e-4dcd-9922-1a7e21e6a8ce 018095d55e3b4487b2a4468c438c3729 11df26b18a1649358ecc4faa64f83262 - default default] Volume ea420554-66d3-4fc5-904e-9683644c7aec: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ea420554-66d3-4fc5-904e-9683644c7aec'} >2018-08-23 08:06:39.428 505523 WARNING cinder.db.sqlalchemy.api [req-5ae9de1f-d720-402a-b14e-410178c60eba 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-23 08:06:39.840 505523 INFO cinder.volume.manager [req-5ae9de1f-d720-402a-b14e-410178c60eba 8485479ee62348e4ad7995c7f284516a df4cfcc3a30541b4abeb3c0702b06390 - default default] Deleted volume successfully. >2018-08-23 08:06:40.140 505464 INFO cinder.volume.flows.manager.create_volume [req-41ead9b8-017e-4dcd-9922-1a7e21e6a8ce 018095d55e3b4487b2a4468c438c3729 11df26b18a1649358ecc4faa64f83262 - default default] Volume volume-ea420554-66d3-4fc5-904e-9683644c7aec (ea420554-66d3-4fc5-904e-9683644c7aec): created successfully >2018-08-23 08:06:40.144 505464 INFO cinder.volume.manager [req-41ead9b8-017e-4dcd-9922-1a7e21e6a8ce 018095d55e3b4487b2a4468c438c3729 11df26b18a1649358ecc4faa64f83262 - default default] Created volume successfully. >2018-08-23 08:06:42.797 505464 INFO cinder.volume.manager [req-3d532d32-434c-4e59-b9fa-558d25d5052d 018095d55e3b4487b2a4468c438c3729 11df26b18a1649358ecc4faa64f83262 - default default] Deleted volume successfully. >2018-08-23 08:06:43.865 505523 INFO cinder.volume.flows.manager.create_volume [req-e3fb1d9f-dbb4-473f-a7d8-611ddefcce43 aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Volume 747da0fe-585f-4562-ace1-a02e1b761089: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-747da0fe-585f-4562-ace1-a02e1b761089'} >2018-08-23 08:06:44.394 505523 INFO cinder.volume.flows.manager.create_volume [req-e3fb1d9f-dbb4-473f-a7d8-611ddefcce43 aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Volume volume-747da0fe-585f-4562-ace1-a02e1b761089 (747da0fe-585f-4562-ace1-a02e1b761089): created successfully >2018-08-23 08:06:44.397 505523 INFO cinder.volume.manager [req-e3fb1d9f-dbb4-473f-a7d8-611ddefcce43 aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Created volume successfully. >2018-08-23 08:06:47.509 505523 INFO cinder.volume.manager [req-35f614fc-a5bb-43a2-b862-b5ca6071633c aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Create snapshot completed successfully >2018-08-23 08:06:49.559 505523 INFO cinder.volume.manager [req-e959cf67-d510-49fd-8871-3bfe15799c9f aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Create snapshot completed successfully >2018-08-23 08:06:51.501 505464 INFO cinder.volume.flows.manager.create_volume [req-6f017e47-b9e0-44e5-a236-fc180900216d 027cac25d109418b8739c2fba0314054 37462be225b94b709e101ccb5eb16da2 - default default] Volume 4cd1af42-8bbc-4646-99c5-adf657a1eeb3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4cd1af42-8bbc-4646-99c5-adf657a1eeb3'} >2018-08-23 08:06:52.413 505464 INFO cinder.volume.flows.manager.create_volume [req-6f017e47-b9e0-44e5-a236-fc180900216d 027cac25d109418b8739c2fba0314054 37462be225b94b709e101ccb5eb16da2 - default default] Volume volume-4cd1af42-8bbc-4646-99c5-adf657a1eeb3 (4cd1af42-8bbc-4646-99c5-adf657a1eeb3): created successfully >2018-08-23 08:06:52.416 505464 INFO cinder.volume.manager [req-6f017e47-b9e0-44e5-a236-fc180900216d 027cac25d109418b8739c2fba0314054 37462be225b94b709e101ccb5eb16da2 - default default] Created volume successfully. >2018-08-23 08:06:52.795 505523 INFO cinder.volume.manager [req-8b2c0380-e52b-4423-8c94-12eaf0d34a3f aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Create snapshot completed successfully >2018-08-23 08:06:54.080 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 08:06:54.177 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 08:06:57.260 505464 WARNING cinder.quota [req-5e493ebe-108f-4601-8845-4f972f005003 027cac25d109418b8739c2fba0314054 37462be225b94b709e101ccb5eb16da2 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1569587037 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1569587037, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 08:06:57.261 505464 WARNING cinder.quota [req-5e493ebe-108f-4601-8845-4f972f005003 027cac25d109418b8739c2fba0314054 37462be225b94b709e101ccb5eb16da2 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1569587037 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1569587037, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 08:06:58.117 505523 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 08:06:58.166 505464 INFO cinder.volume.manager [req-5e493ebe-108f-4601-8845-4f972f005003 027cac25d109418b8739c2fba0314054 37462be225b94b709e101ccb5eb16da2 - default default] Deleted volume successfully. >2018-08-23 08:06:58.207 505523 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 08:06:59.909 505523 INFO cinder.volume.manager [req-3ee53870-beb6-443b-9abd-e5384e1d9277 aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Delete snapshot completed successfully. >2018-08-23 08:07:00.637 505523 INFO cinder.volume.manager [req-83e7d1ad-b597-4904-88f8-789d3ab8d78f aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Delete snapshot completed successfully. >2018-08-23 08:07:01.163 505523 INFO cinder.volume.manager [req-4f49ab4d-3dd2-4720-9e47-eb9443493fd1 aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Delete snapshot completed successfully. >2018-08-23 08:07:03.487 505523 INFO cinder.volume.manager [req-25a8d474-a75a-48c5-acdc-1d63482721c4 aa661e67f1824cacab3e1f182ccb3ed1 6675a3acdf8f482b89cd3340258e7101 - default default] Deleted volume successfully. >2018-08-23 08:07:06.794 505523 INFO cinder.volume.flows.manager.create_volume [req-5e5acb19-558c-4915-bb29-1940096d5132 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] Volume 0e932cc0-469d-4c62-b244-99fce4695bbb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0e932cc0-469d-4c62-b244-99fce4695bbb'} >2018-08-23 08:07:07.657 505523 INFO cinder.volume.flows.manager.create_volume [req-5e5acb19-558c-4915-bb29-1940096d5132 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] Volume volume-0e932cc0-469d-4c62-b244-99fce4695bbb (0e932cc0-469d-4c62-b244-99fce4695bbb): created successfully >2018-08-23 08:07:07.661 505523 INFO cinder.volume.manager [req-5e5acb19-558c-4915-bb29-1940096d5132 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] Created volume successfully. >2018-08-23 08:07:08.920 505523 INFO cinder.volume.manager [req-c37c07b1-522c-41a9-83ba-0a5a0f39eefc 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] Retype volume completed successfully. >2018-08-23 08:07:09.320 505523 INFO cinder.volume.flows.manager.create_volume [req-148b8e19-9606-4bb5-b72f-992d20d7b89b 2d796e07bedd4af2a369e9254a252530 ed4297f6a2e14928af321371b2e33a43 - default default] Volume ee560321-724f-4ca8-9325-8aad4dcd9496: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-ee560321-724f-4ca8-9325-8aad4dcd9496'} >2018-08-23 08:07:10.180 505523 INFO cinder.volume.flows.manager.create_volume [req-148b8e19-9606-4bb5-b72f-992d20d7b89b 2d796e07bedd4af2a369e9254a252530 ed4297f6a2e14928af321371b2e33a43 - default default] Volume volume-ee560321-724f-4ca8-9325-8aad4dcd9496 (ee560321-724f-4ca8-9325-8aad4dcd9496): created successfully >2018-08-23 08:07:10.184 505523 INFO cinder.volume.manager [req-148b8e19-9606-4bb5-b72f-992d20d7b89b 2d796e07bedd4af2a369e9254a252530 ed4297f6a2e14928af321371b2e33a43 - default default] Created volume successfully. >2018-08-23 08:07:13.532 505523 WARNING cinder.quota [req-b4b6fe2f-be3e-46b3-958c-b9ab2c56150e 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1043349637 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1043349637, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 08:07:13.533 505523 WARNING cinder.quota [req-b4b6fe2f-be3e-46b3-958c-b9ab2c56150e 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1043349637 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1043349637, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 08:07:13.987 505523 INFO cinder.volume.manager [req-b4b6fe2f-be3e-46b3-958c-b9ab2c56150e 0b80afb58a1f4203869a9ad6fe6f0eb5 078357eb11284269846cb3d4b4013604 - default default] Deleted volume successfully. >2018-08-23 08:07:19.587 505523 INFO cinder.volume.manager [req-24fd288c-9b5e-4eb8-a721-4de4086c9ba0 2d796e07bedd4af2a369e9254a252530 ed4297f6a2e14928af321371b2e33a43 - default default] Create snapshot completed successfully >2018-08-23 08:07:23.313 505523 INFO cinder.volume.manager [req-ded286a2-8c91-4f97-bf44-9d1e135c5117 2d796e07bedd4af2a369e9254a252530 ed4297f6a2e14928af321371b2e33a43 - default default] Delete snapshot completed successfully. >2018-08-23 08:07:25.544 505523 INFO cinder.volume.manager [req-382dd29e-b488-4bfb-951b-a9ef37f9c861 2d796e07bedd4af2a369e9254a252530 ed4297f6a2e14928af321371b2e33a43 - default default] Deleted volume successfully. >2018-08-23 08:10:40.203 505017 INFO oslo_service.service [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Caught SIGTERM, stopping children >2018-08-23 08:10:40.210 505017 INFO oslo_service.service [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Waiting on 2 children to exit >2018-08-23 08:10:52.282 505464 WARNING py.warnings [-] /usr/lib/python2.7/site-packages/amqp/connection.py:304: AMQPDeprecationWarning: The .transport attribute on the connection was accessed before >the connection was established. This is supported for now, but will >be deprecated in amqp 2.2.0. > >Since amqp 2.0 you have to explicitly call Connection.connect() >before using the connection. > > W_FORCE_CONNECT.format(attr=attr))) >: ConnectionForced: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' >2018-08-23 08:10:52.400 505464 ERROR oslo.messaging._drivers.impl_rabbit [-] [27e6c0c7-15b0-4233-b605-1c863f12440e] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'. Trying again in 1 seconds. Client port: None: ConnectionForced: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' >2018-08-23 08:10:53.522 505464 ERROR oslo.messaging._drivers.impl_rabbit [-] [27e6c0c7-15b0-4233-b605-1c863f12440e] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 08:10:55.642 505464 ERROR oslo.messaging._drivers.impl_rabbit [-] [27e6c0c7-15b0-4233-b605-1c863f12440e] AMQP server on overcloud-controller-0.internalapi.localdomain:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds. Client port: None: error: [Errno 111] ECONNREFUSED >2018-08-23 08:10:55.654 505464 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 08:10:56.515 505464 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 08:10:57.752 505523 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer >2018-08-23 08:10:58.043 505017 INFO oslo_service.service [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Child 505464 exited with status 0 >2018-08-23 08:10:58.075 505017 INFO oslo_service.service [req-3ae71f6f-6095-4809-8632-089208eb2a17 - - - - -] Child 505523 exited with status 0 >2018-08-23 08:38:58.300 7586 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 08:38:58.758 7586 WARNING oslo_db.sqlalchemy.engines [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] SQL connection failed. -1 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.13' ([Errno 111] ECONNREFUSED)") >2018-08-23 08:39:08.770 7586 WARNING oslo_db.sqlalchemy.engines [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] SQL connection failed. -2 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:18.784 7586 WARNING oslo_db.sqlalchemy.engines [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] SQL connection failed. -3 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:28.792 7586 WARNING oslo_db.sqlalchemy.engines [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] SQL connection failed. -4 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:38.800 7586 WARNING oslo_db.sqlalchemy.engines [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] SQL connection failed. -5 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:48.810 7586 WARNING oslo_db.sqlalchemy.engines [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] SQL connection failed. -6 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') >2018-08-23 08:39:58.995 7586 INFO cinder.rpc [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 08:39:59.002 7586 INFO cinder.rpc [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 08:39:59.013 7586 WARNING cinder.keymgr.conf_key_mgr [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-23 08:40:00.465 7586 INFO cinder.volume.manager [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 08:40:00.490 7586 INFO oslo_service.service [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Starting 1 workers >2018-08-23 08:40:00.502 17297 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 08:40:00.521 17297 INFO cinder.volume.manager [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:40:01.522 7586 INFO cinder.volume.manager [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 08:40:01.536 7586 INFO oslo_service.service [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Starting 1 workers >2018-08-23 08:40:01.545 17422 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 08:40:01.545 7586 WARNING oslo_config.cfg [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 08:40:01.553 7586 WARNING oslo_config.cfg [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:01.553 7586 WARNING oslo_config.cfg [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:01.554 7586 WARNING oslo_config.cfg [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:01.561 17422 INFO cinder.volume.manager [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:40:30.550 17297 ERROR cinder.ssh_utils [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 08:40:30.553 17297 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:30.553 17297 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:30.553 17297 ERROR cinder.volume.manager >2018-08-23 08:40:30.592 17297 WARNING oslo_config.cfg [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:30.593 17297 WARNING oslo_config.cfg [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:30.593 17297 WARNING oslo_config.cfg [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:31.566 17422 ERROR cinder.ssh_utils [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 08:40:31.567 17422 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:31.568 17422 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 08:40:31.568 17422 ERROR cinder.volume.manager >2018-08-23 08:40:31.588 17422 WARNING oslo_config.cfg [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:31.588 17422 WARNING oslo_config.cfg [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:40:31.589 17422 WARNING oslo_config.cfg [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:41:30.680 17297 INFO cinder.volume.manager [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:41:30.682 17297 ERROR cinder.utils [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 08:41:30.683 17297 ERROR cinder.volume.manager [req-5fd49c5c-9ff4-4004-9fb2-01538d42d9f3 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 08:41:31.662 17422 INFO cinder.volume.manager [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:41:31.664 17422 ERROR cinder.utils [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 08:41:31.665 17422 ERROR cinder.volume.manager [req-fbb9e7ef-22a5-482f-936f-23f622e6ccae - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 08:41:40.684 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:41:41.667 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:41:50.686 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:41:51.668 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:00.687 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:01.669 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:02.686 17297 WARNING cinder.volume.manager [req-6fb21dbe-e037-4262-a1c1-2a3c2b0ae7a3 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:42:10.688 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:11.668 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:42:11.669 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:20.691 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:21.669 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:30.694 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:31.670 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:40.695 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:41.672 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:50.696 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:42:51.672 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:00.696 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:01.673 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:02.688 17297 WARNING cinder.volume.manager [req-6fb21dbe-e037-4262-a1c1-2a3c2b0ae7a3 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:43:10.698 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:11.670 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:43:11.674 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:20.700 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:21.674 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:30.701 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:31.675 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:40.702 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:41.675 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:50.703 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:43:51.676 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:00.703 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:01.676 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:02.692 17297 WARNING cinder.volume.manager [req-6fb21dbe-e037-4262-a1c1-2a3c2b0ae7a3 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:44:10.704 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:11.672 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:44:11.677 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:20.705 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:21.677 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:30.707 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:31.678 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:40.709 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:41.679 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:50.711 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:44:51.680 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:00.713 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:01.680 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:02.695 17297 WARNING cinder.volume.manager [req-6fb21dbe-e037-4262-a1c1-2a3c2b0ae7a3 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:45:10.715 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:11.674 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:45:11.681 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:20.716 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:21.681 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:30.717 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:31.682 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:40.719 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:41.682 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:50.721 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:45:51.683 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:00.722 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:01.684 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:10.724 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:11.675 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:46:11.686 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:20.724 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:21.686 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:30.726 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:31.687 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:40.728 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:41.688 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:50.729 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:46:51.691 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:00.731 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:01.693 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:02.700 17297 WARNING cinder.volume.manager [req-6fb21dbe-e037-4262-a1c1-2a3c2b0ae7a3 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:47:10.733 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:11.677 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:47:11.694 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:20.736 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:21.695 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:30.738 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:31.696 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:40.740 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:41.696 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:50.741 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:47:51.697 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:00.743 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:01.698 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:02.703 17297 WARNING cinder.volume.manager [req-6fb21dbe-e037-4262-a1c1-2a3c2b0ae7a3 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:48:10.745 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:11.679 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:48:11.699 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:20.746 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:21.699 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:30.747 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:31.700 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:40.747 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:41.700 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:50.748 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:48:51.701 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:49:00.748 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:49:01.702 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:49:02.705 17297 WARNING cinder.volume.manager [req-6fb21dbe-e037-4262-a1c1-2a3c2b0ae7a3 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:49:10.750 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:49:11.681 17422 WARNING cinder.volume.manager [req-1af72cb7-1972-4f1f-87b2-efff9f55ce70 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:49:11.703 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:49:20.752 17297 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:49:21.703 17422 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:49:30.389 7586 INFO oslo_service.service [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Caught SIGTERM, stopping children >2018-08-23 08:49:30.395 7586 INFO oslo_service.service [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Waiting on 2 children to exit >2018-08-23 08:50:02.724 7586 INFO oslo_service.service [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Child 17297 exited with status 0 >2018-08-23 08:50:11.701 7586 INFO oslo_service.service [req-ba1ed474-f2f2-4ca6-a329-8df82b941758 - - - - -] Child 17422 exited with status 0 >2018-08-23 08:50:15.001 78092 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 08:50:15.514 78092 INFO cinder.rpc [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 08:50:15.519 78092 INFO cinder.rpc [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 08:50:15.530 78092 WARNING cinder.keymgr.conf_key_mgr [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-23 08:50:16.575 78092 INFO cinder.volume.manager [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 08:50:16.591 78092 INFO oslo_service.service [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Starting 1 workers >2018-08-23 08:50:16.602 78300 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 08:50:16.618 78300 INFO cinder.volume.manager [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:50:17.622 78092 INFO cinder.volume.manager [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 08:50:17.665 78092 INFO oslo_service.service [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Starting 1 workers >2018-08-23 08:50:17.679 78092 WARNING oslo_config.cfg [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 08:50:17.680 78308 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 08:50:17.688 78092 WARNING oslo_config.cfg [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:17.689 78092 WARNING oslo_config.cfg [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:17.689 78092 WARNING oslo_config.cfg [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:17.713 78308 INFO cinder.volume.manager [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:50:46.624 78300 ERROR cinder.ssh_utils [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 08:50:46.626 78300 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:46.627 78300 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:46.627 78300 ERROR cinder.volume.manager >2018-08-23 08:50:46.646 78300 WARNING oslo_config.cfg [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:46.647 78300 WARNING oslo_config.cfg [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:46.647 78300 WARNING oslo_config.cfg [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:47.721 78308 ERROR cinder.ssh_utils [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 08:50:47.722 78308 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:47.722 78308 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 08:50:47.722 78308 ERROR cinder.volume.manager >2018-08-23 08:50:47.745 78308 WARNING oslo_config.cfg [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:47.746 78308 WARNING oslo_config.cfg [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:50:47.746 78308 WARNING oslo_config.cfg [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 08:51:46.721 78300 INFO cinder.volume.manager [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:51:46.724 78300 ERROR cinder.utils [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 08:51:46.724 78300 ERROR cinder.volume.manager [req-4832fe7a-7ae7-4d54-9f86-e2d19b1cd5a9 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 08:51:47.809 78308 INFO cinder.volume.manager [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 08:51:47.810 78308 ERROR cinder.utils [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 08:51:47.810 78308 ERROR cinder.volume.manager [req-7c367e46-5cc6-4e9c-9fee-404d9e5d5d8a - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 08:51:56.726 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:51:56.729 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:51:57.812 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:06.728 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:07.814 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:15.683 78300 WARNING cinder.volume.manager [req-0e08eab6-2078-445b-9df5-b4e14c759a83 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:52:15.685 78308 WARNING cinder.volume.manager [req-0e08eab6-2078-445b-9df5-b4e14c759a83 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:52:16.729 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:17.815 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:26.730 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:27.816 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:36.731 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:37.816 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:41.814 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:52:46.733 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:47.817 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:56.731 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:52:56.736 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:52:57.818 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:06.736 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:07.819 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:16.738 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:17.819 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:26.742 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:27.820 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:36.745 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:37.821 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:41.818 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:53:46.747 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:47.822 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:56.734 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:53:56.747 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:53:57.823 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:06.748 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:07.824 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:16.749 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:17.826 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:26.750 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:27.828 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:36.751 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:37.829 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:41.821 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:54:46.753 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:47.830 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:56.738 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:54:56.754 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:54:57.831 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:06.756 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:07.831 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:16.759 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:17.832 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:26.760 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:27.833 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:36.762 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:37.834 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:41.824 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:55:46.762 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:47.836 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:56.740 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:55:56.764 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:55:57.837 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:06.764 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:07.838 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:16.765 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:17.840 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:26.767 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:27.840 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:36.768 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:37.841 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:41.827 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:56:46.769 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:47.842 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:56.743 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:56:56.770 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:56:57.843 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:06.771 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:07.844 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:16.772 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:17.845 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:26.775 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:27.846 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:36.776 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:37.846 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:41.829 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:57:46.777 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:47.847 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:56.744 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:57:56.778 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:57:57.848 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:06.778 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:07.848 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:16.779 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:17.849 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:26.782 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:27.850 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:36.783 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:37.851 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:41.831 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:58:46.785 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:47.851 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:56.746 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:58:56.785 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:58:57.852 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:06.786 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:07.853 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:16.787 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:17.854 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:26.790 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:27.855 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:36.790 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:37.856 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:41.835 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 08:59:46.791 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:47.856 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:56.749 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 08:59:56.792 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 08:59:57.857 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:06.792 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:07.858 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:16.795 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:17.859 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:26.796 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:27.859 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:36.798 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:37.860 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:41.837 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:00:46.801 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:47.861 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:56.751 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:00:56.802 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:00:57.861 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:06.802 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:07.862 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:16.805 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:17.863 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:26.807 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:27.864 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:36.808 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:37.865 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:41.841 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:01:46.809 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:47.865 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:56.753 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:01:56.810 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:01:57.866 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:06.811 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:07.867 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:16.812 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:17.869 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:26.814 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:27.870 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:36.817 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:37.870 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:41.845 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:02:46.819 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:47.871 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:56.756 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:02:56.820 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:02:57.872 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:06.820 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:07.873 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:16.822 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:17.873 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:26.824 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:27.874 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:36.826 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:37.874 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:41.850 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:03:46.826 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:47.875 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:56.758 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:03:56.827 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:03:57.876 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:06.827 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:07.876 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:16.829 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:17.877 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:26.830 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:27.878 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:36.831 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:37.879 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:41.843 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:04:46.833 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:47.880 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:56.760 78300 WARNING cinder.volume.manager [req-189bb06a-faf9-46a4-a805-0b2ad01115a8 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:04:56.834 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:04:57.880 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:06.834 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:07.881 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:16.835 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:17.881 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:26.838 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:27.882 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:33.863 78092 INFO oslo_service.service [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Caught SIGTERM, stopping children >2018-08-23 09:05:33.865 78092 INFO oslo_service.service [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Waiting on 2 children to exit >2018-08-23 09:05:36.839 78300 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:37.886 78308 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:05:41.847 78308 WARNING cinder.volume.manager [req-01dc02b0-6569-41e7-8279-d40de9f34afd - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:05:56.779 78092 INFO oslo_service.service [req-5cde3acb-ea62-44f0-899d-cccbbfa0e7ca - - - - -] Child 78300 exited with status 0 >2018-08-23 09:06:33.875 78308 INFO oslo_service.service [-] Parent process has died unexpectedly, exiting >2018-08-23 09:06:35.642 172178 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 09:06:36.114 172178 INFO cinder.rpc [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 09:06:36.120 172178 INFO cinder.rpc [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 09:06:36.130 172178 INFO cinder.volume.manager [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Service not found for updating active_backend_id, assuming default for driver init. >2018-08-23 09:06:36.131 172178 WARNING cinder.keymgr.conf_key_mgr [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] This key manager is insecure and is not recommended for production deployments: ServiceNotFound: Service cinder-volume could not be found on host hostgroup@svc1. >2018-08-23 09:06:37.177 172178 INFO cinder.volume.manager [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 09:06:37.194 172178 INFO oslo_service.service [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Starting 1 workers >2018-08-23 09:06:37.205 172271 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:06:37.220 172271 INFO cinder.volume.manager [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Including all resources from host hostgroup@svc1 in cluster None. >2018-08-23 09:06:37.220 172178 INFO cinder.volume.manager [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Service not found for updating active_backend_id, assuming default for driver init. >2018-08-23 09:06:37.263 172271 INFO cinder.volume.manager [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] 0 volumes, 0 consistency groups, 0 generic groups and 0 image volume caches from host hostgroup@svc1 have been included in cluster None. >2018-08-23 09:06:37.264 172271 INFO cinder.volume.manager [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:06:38.226 172178 INFO cinder.volume.manager [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 09:06:38.242 172178 INFO oslo_service.service [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Starting 1 workers >2018-08-23 09:06:38.251 172178 WARNING oslo_config.cfg [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 09:06:38.252 172281 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:06:38.258 172178 WARNING oslo_config.cfg [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:06:38.259 172178 WARNING oslo_config.cfg [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:06:38.259 172178 WARNING oslo_config.cfg [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:06:38.267 172281 INFO cinder.volume.manager [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Including all resources from host hostgroup@svc2 in cluster None. >2018-08-23 09:06:38.293 172281 INFO cinder.volume.manager [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] 5 volumes, 0 consistency groups, 0 generic groups and 0 image volume caches from host hostgroup@svc2 have been included in cluster None. >2018-08-23 09:06:38.294 172281 INFO cinder.volume.manager [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:07:07.270 172271 ERROR cinder.ssh_utils [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 09:07:07.271 172271 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:07.271 172271 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:07.272 172271 ERROR cinder.volume.manager >2018-08-23 09:07:07.288 172271 WARNING oslo_config.cfg [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:07.288 172271 WARNING oslo_config.cfg [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:07.289 172271 WARNING oslo_config.cfg [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:08.300 172281 ERROR cinder.ssh_utils [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 09:07:08.301 172281 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:08.301 172281 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 09:07:08.302 172281 ERROR cinder.volume.manager >2018-08-23 09:07:08.316 172281 WARNING oslo_config.cfg [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:08.316 172281 WARNING oslo_config.cfg [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:07:08.316 172281 WARNING oslo_config.cfg [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:08:07.377 172271 INFO cinder.volume.manager [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:08:07.379 172271 ERROR cinder.utils [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 09:08:07.380 172271 ERROR cinder.volume.manager [req-45cd0bfd-c028-4c21-ab36-4bb696e4d107 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 09:08:08.379 172281 INFO cinder.volume.manager [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:08:08.381 172281 ERROR cinder.utils [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 09:08:08.381 172281 ERROR cinder.volume.manager [req-fed62fa2-afd8-42cc-9aef-d51ba9b89a93 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 09:08:17.381 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:18.382 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:27.383 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:28.384 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:36.383 172271 WARNING cinder.volume.manager [req-66139ca6-58a1-44ea-9009-b0179369e463 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:08:37.385 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:38.384 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:41.383 172281 WARNING cinder.volume.manager [req-366384d5-0cfa-4f7a-bd0f-456ae3691c4f - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:08:47.386 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:48.385 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:57.387 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:58.386 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:08:58.467 172281 WARNING cinder.volume.manager [req-731fb67d-8255-45d4-9223-4492795e898a - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:08:58.468 172271 WARNING cinder.volume.manager [req-731fb67d-8255-45d4-9223-4492795e898a - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:09:07.389 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:08.386 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:17.391 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:18.387 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:27.393 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:28.388 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:36.385 172271 WARNING cinder.volume.manager [req-66139ca6-58a1-44ea-9009-b0179369e463 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:09:37.395 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:38.389 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:41.384 172281 WARNING cinder.volume.manager [req-366384d5-0cfa-4f7a-bd0f-456ae3691c4f - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:09:47.395 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:48.390 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:57.398 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:09:58.391 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:10:07.399 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:10:08.392 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:10:17.305 172178 INFO oslo_service.service [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Caught SIGTERM, stopping children >2018-08-23 09:10:17.311 172178 INFO oslo_service.service [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Waiting on 2 children to exit >2018-08-23 09:10:17.400 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:10:18.393 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:10:27.401 172271 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:10:28.394 172281 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:10:41.404 172178 INFO oslo_service.service [req-8765b762-4867-4b52-b018-921a1aa1e971 - - - - -] Child 172281 exited with status 0 >2018-08-23 09:11:17.321 172271 INFO oslo_service.service [-] Parent process has died unexpectedly, exiting >2018-08-23 09:11:20.158 199693 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 09:11:20.682 199693 INFO cinder.rpc [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 09:11:20.687 199693 INFO cinder.rpc [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 09:11:20.697 199693 WARNING cinder.keymgr.conf_key_mgr [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-23 09:11:21.768 199693 INFO cinder.volume.manager [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 09:11:21.788 199693 INFO oslo_service.service [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Starting 1 workers >2018-08-23 09:11:21.798 200100 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:11:21.816 200100 INFO cinder.volume.manager [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:11:22.817 199693 INFO cinder.volume.manager [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 09:11:22.830 199693 INFO oslo_service.service [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Starting 1 workers >2018-08-23 09:11:22.839 199693 WARNING oslo_config.cfg [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 09:11:22.840 200367 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:11:22.846 199693 WARNING oslo_config.cfg [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:22.846 199693 WARNING oslo_config.cfg [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:22.847 199693 WARNING oslo_config.cfg [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:22.856 200367 INFO cinder.volume.manager [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:11:51.822 200100 ERROR cinder.ssh_utils [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 09:11:51.823 200100 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:51.824 200100 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:51.824 200100 ERROR cinder.volume.manager >2018-08-23 09:11:51.848 200100 WARNING oslo_config.cfg [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:51.848 200100 WARNING oslo_config.cfg [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:51.849 200100 WARNING oslo_config.cfg [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:52.862 200367 ERROR cinder.ssh_utils [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 09:11:52.863 200367 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:52.864 200367 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 09:11:52.864 200367 ERROR cinder.volume.manager >2018-08-23 09:11:52.888 200367 WARNING oslo_config.cfg [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:52.888 200367 WARNING oslo_config.cfg [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:11:52.889 200367 WARNING oslo_config.cfg [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:12:51.920 200100 INFO cinder.volume.manager [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:12:51.923 200100 ERROR cinder.utils [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 09:12:51.923 200100 ERROR cinder.volume.manager [req-2fe5ec7b-4105-4f4a-abe8-6d51b0929316 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 09:12:52.925 200100 WARNING cinder.volume.manager [req-b4fa4a71-6485-4916-9691-c68877d1f644 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:12:52.964 200367 INFO cinder.volume.manager [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:12:52.965 200367 ERROR cinder.utils [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 09:12:52.965 200367 ERROR cinder.volume.manager [req-f88e5f8a-0a10-4cc9-b834-786dcbcede1e - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 09:13:01.925 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:02.967 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:11.927 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:12.969 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:21.929 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:22.970 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:31.930 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:32.970 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:38.969 200367 WARNING cinder.volume.manager [req-21ef2f88-34e1-4d60-8334-25586e2623f8 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:13:41.931 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:42.971 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:51.931 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:13:52.928 200100 WARNING cinder.volume.manager [req-b4fa4a71-6485-4916-9691-c68877d1f644 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:13:52.972 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:01.932 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:02.973 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:11.935 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:12.974 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:21.937 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:22.976 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:31.938 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:32.976 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:38.973 200367 WARNING cinder.volume.manager [req-21ef2f88-34e1-4d60-8334-25586e2623f8 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:14:41.939 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:42.977 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:51.941 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:14:52.929 200100 WARNING cinder.volume.manager [req-b4fa4a71-6485-4916-9691-c68877d1f644 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:14:52.978 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:01.942 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:02.979 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:11.942 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:12.979 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:21.943 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:22.980 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:31.944 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:32.981 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:38.976 200367 WARNING cinder.volume.manager [req-21ef2f88-34e1-4d60-8334-25586e2623f8 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:15:39.030 200367 WARNING cinder.volume.manager [req-ce1209c4-067f-45b9-a7ff-a4697b6e2778 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:15:39.030 200100 WARNING cinder.volume.manager [req-ce1209c4-067f-45b9-a7ff-a4697b6e2778 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:15:41.945 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:42.982 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:51.946 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:15:52.931 200100 WARNING cinder.volume.manager [req-b4fa4a71-6485-4916-9691-c68877d1f644 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:15:52.982 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:01.947 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:02.983 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:11.948 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:12.984 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:21.949 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:22.985 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:31.951 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:32.985 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:38.980 200367 WARNING cinder.volume.manager [req-21ef2f88-34e1-4d60-8334-25586e2623f8 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:16:41.953 200100 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:42.986 200367 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:16:51.333 199693 INFO oslo_service.service [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Caught SIGTERM, stopping children >2018-08-23 09:16:51.338 199693 INFO oslo_service.service [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Waiting on 2 children to exit >2018-08-23 09:16:52.952 199693 INFO oslo_service.service [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Child 200100 exited with status 0 >2018-08-23 09:17:39.002 199693 INFO oslo_service.service [req-f21a4068-bd28-41c9-9560-6e0c23a85458 - - - - -] Child 200367 exited with status 0 >2018-08-23 09:17:42.668 237341 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 09:17:43.214 237341 INFO cinder.rpc [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 09:17:43.220 237341 INFO cinder.rpc [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 09:17:43.231 237341 WARNING cinder.keymgr.conf_key_mgr [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-23 09:17:44.299 237341 INFO cinder.volume.manager [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 09:17:44.320 237341 INFO oslo_service.service [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Starting 1 workers >2018-08-23 09:17:44.331 237443 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:17:44.350 237443 INFO cinder.volume.manager [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:17:45.351 237341 INFO cinder.volume.manager [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 09:17:45.367 237341 INFO oslo_service.service [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Starting 1 workers >2018-08-23 09:17:45.377 237341 WARNING oslo_config.cfg [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 09:17:45.378 237582 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:17:45.384 237341 WARNING oslo_config.cfg [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:17:45.385 237341 WARNING oslo_config.cfg [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:17:45.385 237341 WARNING oslo_config.cfg [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:17:45.394 237582 INFO cinder.volume.manager [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:18:14.357 237443 ERROR cinder.ssh_utils [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 09:18:14.360 237443 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:14.360 237443 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:14.360 237443 ERROR cinder.volume.manager >2018-08-23 09:18:14.380 237443 WARNING oslo_config.cfg [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:18:14.380 237443 WARNING oslo_config.cfg [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:18:14.381 237443 WARNING oslo_config.cfg [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:18:14.663 237443 INFO cinder.volume.manager [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:18:14.665 237443 ERROR cinder.utils [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 09:18:14.665 237443 ERROR cinder.volume.manager [req-2a192296-738e-4e7e-8c54-4bed5105258a - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 09:18:15.399 237582 ERROR cinder.ssh_utils [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-23 09:18:15.400 237582 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Unable to use san_ip to create SSHPool. Now attempting to use storwize_san_secondary_ip to create SSHPool.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:15.401 237582 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Unable to create SSHPool using san_ip and not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Failed to initialize driver.: SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 430, in init_host >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager self.driver.do_setup(ctxt) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2236, in do_setup >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager self._get_storwize_config() >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3252, in _get_storwize_config >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager self._replica_enabled = (True if (self._helpers.replication_licensed() >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 694, in replication_licensed >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager resp = self.ssh.lsguicapabilities() >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 204, in lsguicapabilities >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd)[0] >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2367, in _run_ssh >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager self.sshpool = self._set_up_sshpool(self.active_ip) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2420, in _set_up_sshpool >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager max_size=max_size) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 100, in __init__ >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager super(SSHPool, self).__init__(*args, **kwargs) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 77, in __init__ >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager self.free_items.append(self.create()) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-23 09:18:15.401 237582 ERROR cinder.volume.manager >2018-08-23 09:18:15.421 237582 WARNING oslo_config.cfg [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:18:15.422 237582 WARNING oslo_config.cfg [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:18:15.422 237582 WARNING oslo_config.cfg [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:18:15.698 237582 INFO cinder.volume.manager [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:18:15.698 237582 ERROR cinder.utils [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Volume driver StorwizeSVCFCDriver not initialized >2018-08-23 09:18:15.699 237582 ERROR cinder.volume.manager [req-5f919a77-46fb-438e-bb9b-1d169fbdd1b5 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. >2018-08-23 09:18:24.666 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:18:25.700 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:18:34.439 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:18:35.472 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:18:43.440 237443 WARNING cinder.volume.manager [req-c1cf691a-a03e-4fe5-8681-838cf2d593d0 - - - - -] Update driver status failed: (config name svc1) is uninitialized. >2018-08-23 09:18:44.440 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:18:45.473 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:18:54.441 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:18:55.475 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:01.472 237582 WARNING cinder.volume.manager [req-ca62646a-5495-444e-82c6-b5be4cbdcc86 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:19:04.443 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:05.477 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:14.444 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:15.477 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:24.446 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:25.478 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:34.448 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:35.480 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:44.450 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:45.480 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:54.450 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:19:55.481 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:20:01.475 237582 WARNING cinder.volume.manager [req-ca62646a-5495-444e-82c6-b5be4cbdcc86 - - - - -] Update driver status failed: (config name svc2) is uninitialized. >2018-08-23 09:20:04.452 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:20:05.482 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:20:14.454 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:20:15.484 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:20:24.456 237443 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc1 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:20:25.486 237582 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@svc2 is reporting problems, not sending heartbeat. Service will appear "down". >2018-08-23 09:20:26.804 237341 INFO oslo_service.service [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Caught SIGTERM, stopping children >2018-08-23 09:20:26.807 237341 INFO oslo_service.service [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Waiting on 2 children to exit >2018-08-23 09:20:43.473 237341 INFO oslo_service.service [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Child 237443 exited with status 0 >2018-08-23 09:21:01.496 237341 INFO oslo_service.service [req-597b7883-4519-4e59-ae58-306ef8bd3c02 - - - - -] Child 237582 exited with status 0 >2018-08-23 09:21:04.835 256255 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-23 09:21:05.311 256255 INFO cinder.rpc [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-23 09:21:05.316 256255 INFO cinder.rpc [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-23 09:21:05.326 256255 WARNING cinder.keymgr.conf_key_mgr [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-23 09:21:06.371 256255 INFO cinder.volume.manager [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-23 09:21:06.386 256255 INFO oslo_service.service [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Starting 1 workers >2018-08-23 09:21:06.394 257031 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:21:06.409 257031 INFO cinder.volume.manager [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:21:06.424 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 09:21:06.482 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 09:21:07.413 256255 INFO cinder.volume.manager [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-23 09:21:07.428 256255 INFO oslo_service.service [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Starting 1 workers >2018-08-23 09:21:07.436 256255 WARNING oslo_config.cfg [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 09:21:07.437 257256 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-23 09:21:07.443 256255 WARNING oslo_config.cfg [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:07.444 256255 WARNING oslo_config.cfg [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:07.444 256255 WARNING oslo_config.cfg [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:07.454 257256 INFO cinder.volume.manager [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Starting volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:21:07.476 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 09:21:07.547 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 09:21:09.206 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] WWPN on node 158: [u'500507680C384AFD', u'500507680C364AFD', u'500507680C114AFD', u'500507680C344AFD', u'500507680C324AFD', u'500507680C174AFD', u'500507680C334AFD', u'500507680C134AFD', u'500507680C154AFD', u'500507680C124AFD', u'500507680C354AFD', u'500507680C314AFD', u'500507680C184AFD', u'500507680C374AFD', u'500507680C144AFD', u'500507680C164AFD']. >2018-08-23 09:21:09.595 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] WWPN on node 157: [u'500507680C1648C2', u'500507680C3648C2', u'500507680C3748C2', u'500507680C3848C2', u'500507680C1348C2', u'500507680C1848C2', u'500507680C3348C2', u'500507680C3548C2', u'500507680C1448C2', u'500507680C1548C2', u'500507680C1148C2', u'500507680C1748C2', u'500507680C1248C2', u'500507680C3248C2', u'500507680C3448C2', u'500507680C3148C2']. >2018-08-23 09:21:10.594 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] WWPN on node 158: [u'500507680C384AFD', u'500507680C364AFD', u'500507680C114AFD', u'500507680C344AFD', u'500507680C324AFD', u'500507680C174AFD', u'500507680C334AFD', u'500507680C134AFD', u'500507680C154AFD', u'500507680C124AFD', u'500507680C354AFD', u'500507680C314AFD', u'500507680C184AFD', u'500507680C374AFD', u'500507680C144AFD', u'500507680C164AFD']. >2018-08-23 09:21:10.867 257031 WARNING oslo_config.cfg [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:10.868 257031 WARNING oslo_config.cfg [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:10.868 257031 WARNING oslo_config.cfg [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:10.973 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] WWPN on node 157: [u'500507680C1648C2', u'500507680C3648C2', u'500507680C3748C2', u'500507680C3848C2', u'500507680C1348C2', u'500507680C1848C2', u'500507680C3348C2', u'500507680C3548C2', u'500507680C1448C2', u'500507680C1548C2', u'500507680C1148C2', u'500507680C1748C2', u'500507680C1248C2', u'500507680C3248C2', u'500507680C3448C2', u'500507680C3148C2']. >2018-08-23 09:21:11.005 257031 INFO cinder.volume.manager [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Driver initialization completed successfully. >2018-08-23 09:21:11.013 257031 INFO cinder.manager [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Initiating service 6 cleanup >2018-08-23 09:21:11.016 257031 INFO cinder.manager [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Service 6 cleanup completed. >2018-08-23 09:21:11.298 257031 INFO cinder.volume.manager [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:21:11.698 257031 INFO cinder.volume.manager [req-227c209c-1a33-4c52-9391-6b357490a515 - - - - -] Driver post RPC initialization completed successfully. >2018-08-23 09:21:11.940 257256 WARNING oslo_config.cfg [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:11.940 257256 WARNING oslo_config.cfg [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:11.941 257256 WARNING oslo_config.cfg [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-23 09:21:12.076 257256 INFO cinder.volume.manager [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Driver initialization completed successfully. >2018-08-23 09:21:12.082 257256 INFO cinder.manager [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Initiating service 7 cleanup >2018-08-23 09:21:12.086 257256 INFO cinder.manager [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Service 7 cleanup completed. >2018-08-23 09:21:12.366 257256 INFO cinder.volume.manager [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCFCDriver (2.2.2) >2018-08-23 09:21:12.594 257256 INFO cinder.volume.manager [req-ca74b89f-5094-407b-a284-3c88df67ff99 - - - - -] Driver post RPC initialization completed successfully. >2018-08-23 09:22:03.045 257256 INFO cinder.volume.flows.manager.create_volume [req-e0efc548-4c75-4251-9d3f-8727df38116e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume 3471e841-62e9-448a-898d-39885a3ee4a8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3471e841-62e9-448a-898d-39885a3ee4a8'} >2018-08-23 09:22:03.690 257256 INFO cinder.volume.flows.manager.create_volume [req-e0efc548-4c75-4251-9d3f-8727df38116e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume volume-3471e841-62e9-448a-898d-39885a3ee4a8 (3471e841-62e9-448a-898d-39885a3ee4a8): created successfully >2018-08-23 09:22:03.694 257256 INFO cinder.volume.manager [req-e0efc548-4c75-4251-9d3f-8727df38116e e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Created volume successfully. >2018-08-23 09:22:15.114 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-8d8f007c-bfcd-4372-8981-a71bd056cb51 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Tried to delete non-existent vdisk volume-157398ef-7ee3-4092-8155-b9bda081955c. >2018-08-23 09:22:15.416 257256 INFO cinder.volume.manager [req-8d8f007c-bfcd-4372-8981-a71bd056cb51 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deleted volume successfully. >2018-08-23 09:22:21.679 257256 INFO cinder.volume.manager [req-803d10c4-2443-4944-bb32-fd9b086c5ea3 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deleted volume successfully. >2018-08-23 09:23:15.783 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-8a6865fe-95da-4968-b0bf-709c5266dd15 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Tried to delete non-existent vdisk volume-97ee898a-1ab7-45a3-8e7d-5879f62741b6. >2018-08-23 09:23:16.414 257256 INFO cinder.volume.manager [req-8a6865fe-95da-4968-b0bf-709c5266dd15 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deleted volume successfully. >2018-08-23 09:29:27.159 257256 INFO cinder.volume.flows.manager.create_volume [req-fec414e8-1e15-4eee-9e16-48b5b8ff3b96 8d5b4879010a463e9b89426492731404 0882cb703fa54d459c22665a7aede24e - default default] Volume 80b1dcdd-0bfe-494e-b83e-fd338bb32df1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-80b1dcdd-0bfe-494e-b83e-fd338bb32df1'} >2018-08-23 09:29:27.746 257256 INFO cinder.volume.flows.manager.create_volume [req-fec414e8-1e15-4eee-9e16-48b5b8ff3b96 8d5b4879010a463e9b89426492731404 0882cb703fa54d459c22665a7aede24e - default default] Volume volume-80b1dcdd-0bfe-494e-b83e-fd338bb32df1 (80b1dcdd-0bfe-494e-b83e-fd338bb32df1): created successfully >2018-08-23 09:29:27.750 257256 INFO cinder.volume.manager [req-fec414e8-1e15-4eee-9e16-48b5b8ff3b96 8d5b4879010a463e9b89426492731404 0882cb703fa54d459c22665a7aede24e - default default] Created volume successfully. >2018-08-23 09:29:32.000 257256 INFO cinder.volume.manager [req-ee77ed7b-2c8d-4ba8-8853-d2e3aa3ad649 8d5b4879010a463e9b89426492731404 0882cb703fa54d459c22665a7aede24e - default default] Deleted volume successfully. >2018-08-23 09:29:41.811 257256 INFO cinder.volume.flows.manager.create_volume [req-5dae75fc-b138-4288-bdef-a111776a797c d1127a4b484a4467acc1dcc1a902d600 268d51af585a4706b313752439498cef - default default] Volume 11f3b5b1-eba3-4f62-a100-ed5f3058d51c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-11f3b5b1-eba3-4f62-a100-ed5f3058d51c'} >2018-08-23 09:29:42.416 257256 INFO cinder.volume.flows.manager.create_volume [req-5dae75fc-b138-4288-bdef-a111776a797c d1127a4b484a4467acc1dcc1a902d600 268d51af585a4706b313752439498cef - default default] Volume volume-11f3b5b1-eba3-4f62-a100-ed5f3058d51c (11f3b5b1-eba3-4f62-a100-ed5f3058d51c): created successfully >2018-08-23 09:29:42.420 257256 INFO cinder.volume.manager [req-5dae75fc-b138-4288-bdef-a111776a797c d1127a4b484a4467acc1dcc1a902d600 268d51af585a4706b313752439498cef - default default] Created volume successfully. >2018-08-23 09:29:47.245 257256 WARNING cinder.quota [req-e2146046-064f-4a57-876b-f5ede96a3f57 d1127a4b484a4467acc1dcc1a902d600 268d51af585a4706b313752439498cef - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1352586942 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1352586942, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:29:47.246 257256 WARNING cinder.quota [req-e2146046-064f-4a57-876b-f5ede96a3f57 d1127a4b484a4467acc1dcc1a902d600 268d51af585a4706b313752439498cef - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1352586942 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1352586942, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:29:47.547 257256 INFO cinder.volume.manager [req-e2146046-064f-4a57-876b-f5ede96a3f57 d1127a4b484a4467acc1dcc1a902d600 268d51af585a4706b313752439498cef - default default] Deleted volume successfully. >2018-08-23 09:29:57.739 257256 INFO cinder.volume.flows.manager.create_volume [req-50446ffe-4169-4594-8ec2-a28e64a00f52 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] Volume e8c03702-a4a2-44b5-9899-d6334563978d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e8c03702-a4a2-44b5-9899-d6334563978d'} >2018-08-23 09:29:58.383 257256 INFO cinder.volume.flows.manager.create_volume [req-50446ffe-4169-4594-8ec2-a28e64a00f52 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] Volume volume-e8c03702-a4a2-44b5-9899-d6334563978d (e8c03702-a4a2-44b5-9899-d6334563978d): created successfully >2018-08-23 09:29:58.387 257256 INFO cinder.volume.manager [req-50446ffe-4169-4594-8ec2-a28e64a00f52 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] Created volume successfully. >2018-08-23 09:29:59.877 257256 INFO cinder.volume.manager [req-d03647d4-cc29-493c-be4f-c2fc285b7af0 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] Retype volume completed successfully. >2018-08-23 09:30:04.347 257256 WARNING cinder.quota [req-c2b88b1a-b14a-47e2-a9d7-6b9e0e40b641 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1375135014 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1375135014, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:30:04.348 257256 WARNING cinder.quota [req-c2b88b1a-b14a-47e2-a9d7-6b9e0e40b641 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1375135014 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1375135014, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:30:04.595 257256 INFO cinder.volume.manager [req-c2b88b1a-b14a-47e2-a9d7-6b9e0e40b641 53339db610784a8083e73c06b04715e1 98e261d113f54e479e7e55abd1c7bd20 - default default] Deleted volume successfully. >2018-08-23 09:30:49.146 257256 INFO cinder.volume.flows.manager.create_volume [req-666e981f-d681-44b6-8a24-609b462a1447 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume f93953ba-22b1-4d57-bf67-41094691421f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f93953ba-22b1-4d57-bf67-41094691421f'} >2018-08-23 09:30:49.744 257256 INFO cinder.volume.flows.manager.create_volume [req-666e981f-d681-44b6-8a24-609b462a1447 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume volume-f93953ba-22b1-4d57-bf67-41094691421f (f93953ba-22b1-4d57-bf67-41094691421f): created successfully >2018-08-23 09:30:49.748 257256 INFO cinder.volume.manager [req-666e981f-d681-44b6-8a24-609b462a1447 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Created volume successfully. >2018-08-23 09:30:52.656 257256 INFO cinder.volume.manager [req-33279946-0ab1-4ed4-a918-a4fdef9baec8 32c1ded2f1364cf78e2b7fcd5ec1c52d bac9ae940d5e415eae3702139f97936f - default default] Deleted volume successfully. >2018-08-23 09:30:53.311 257256 INFO cinder.volume.flows.manager.create_volume [req-ca903889-2d16-49a1-800f-9a6b71f8a223 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume 030bb09a-e7dc-446f-80f7-dad60472e346: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-030bb09a-e7dc-446f-80f7-dad60472e346'} >2018-08-23 09:30:53.913 257256 INFO cinder.volume.flows.manager.create_volume [req-ca903889-2d16-49a1-800f-9a6b71f8a223 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume volume-030bb09a-e7dc-446f-80f7-dad60472e346 (030bb09a-e7dc-446f-80f7-dad60472e346): created successfully >2018-08-23 09:30:53.916 257256 INFO cinder.volume.manager [req-ca903889-2d16-49a1-800f-9a6b71f8a223 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Created volume successfully. >2018-08-23 09:30:56.308 257256 INFO cinder.volume.manager [req-39455b85-028b-4af5-aac3-635b1f5cb945 32c1ded2f1364cf78e2b7fcd5ec1c52d bac9ae940d5e415eae3702139f97936f - default default] Deleted volume successfully. >2018-08-23 09:30:56.562 257256 INFO cinder.volume.flows.manager.create_volume [req-1744496a-d345-42fb-adde-2b4762415e7e 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume f162c7df-bfd6-4cac-875e-dc0bdb953c35: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f162c7df-bfd6-4cac-875e-dc0bdb953c35'} >2018-08-23 09:30:57.144 257256 INFO cinder.volume.flows.manager.create_volume [req-1744496a-d345-42fb-adde-2b4762415e7e 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume volume-f162c7df-bfd6-4cac-875e-dc0bdb953c35 (f162c7df-bfd6-4cac-875e-dc0bdb953c35): created successfully >2018-08-23 09:30:57.150 257256 INFO cinder.volume.manager [req-1744496a-d345-42fb-adde-2b4762415e7e 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Created volume successfully. >2018-08-23 09:30:59.427 257256 INFO cinder.volume.manager [req-0ba0fffe-9f9e-4af5-b041-fed174e110af 32c1ded2f1364cf78e2b7fcd5ec1c52d bac9ae940d5e415eae3702139f97936f - default default] Deleted volume successfully. >2018-08-23 09:30:59.845 257256 INFO cinder.volume.flows.manager.create_volume [req-22382ecd-631b-4408-bb9b-3c751391084f 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume 1443ffa4-0818-4027-aa10-44b3a24253d6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1443ffa4-0818-4027-aa10-44b3a24253d6'} >2018-08-23 09:31:00.428 257256 INFO cinder.volume.flows.manager.create_volume [req-22382ecd-631b-4408-bb9b-3c751391084f 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume volume-1443ffa4-0818-4027-aa10-44b3a24253d6 (1443ffa4-0818-4027-aa10-44b3a24253d6): created successfully >2018-08-23 09:31:00.432 257256 INFO cinder.volume.manager [req-22382ecd-631b-4408-bb9b-3c751391084f 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Created volume successfully. >2018-08-23 09:31:02.972 257256 INFO cinder.volume.manager [req-2e7367ef-0068-471f-957c-a4ebbeb51c5c 32c1ded2f1364cf78e2b7fcd5ec1c52d bac9ae940d5e415eae3702139f97936f - default default] Deleted volume successfully. >2018-08-23 09:31:03.280 257256 INFO cinder.volume.flows.manager.create_volume [req-68f318e1-c2e1-4857-97d5-27ac2ad2bdcc 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume 64a5a908-d546-4f6d-89df-e646674814f9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-64a5a908-d546-4f6d-89df-e646674814f9'} >2018-08-23 09:31:03.947 257256 INFO cinder.volume.flows.manager.create_volume [req-68f318e1-c2e1-4857-97d5-27ac2ad2bdcc 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Volume volume-64a5a908-d546-4f6d-89df-e646674814f9 (64a5a908-d546-4f6d-89df-e646674814f9): created successfully >2018-08-23 09:31:03.951 257256 INFO cinder.volume.manager [req-68f318e1-c2e1-4857-97d5-27ac2ad2bdcc 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Created volume successfully. >2018-08-23 09:31:07.123 257256 INFO cinder.volume.manager [req-78ed119d-12a7-4fd7-bd4f-fd0d6bee9316 4adce0c56b7242b0a48312fabda45810 14e117a745df44108d74be390f1eea71 - default default] Deleted volume successfully. >2018-08-23 09:31:14.747 257256 INFO cinder.volume.flows.manager.create_volume [req-567751c8-4605-443c-b179-f64cc67fef1e c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Volume 7f6c68c0-e071-4e30-bedc-ec60c5aa7dbd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7f6c68c0-e071-4e30-bedc-ec60c5aa7dbd'} >2018-08-23 09:31:15.333 257256 INFO cinder.volume.flows.manager.create_volume [req-567751c8-4605-443c-b179-f64cc67fef1e c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Volume volume-7f6c68c0-e071-4e30-bedc-ec60c5aa7dbd (7f6c68c0-e071-4e30-bedc-ec60c5aa7dbd): created successfully >2018-08-23 09:31:15.336 257256 INFO cinder.volume.manager [req-567751c8-4605-443c-b179-f64cc67fef1e c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Created volume successfully. >2018-08-23 09:31:16.793 257256 INFO cinder.volume.flows.manager.create_volume [req-63f32b95-2863-4912-aab0-ba60eb254692 c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Volume eff89691-35c0-4adb-aaa5-84e208495de1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-eff89691-35c0-4adb-aaa5-84e208495de1'} >2018-08-23 09:31:17.377 257256 INFO cinder.volume.flows.manager.create_volume [req-63f32b95-2863-4912-aab0-ba60eb254692 c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Volume volume-eff89691-35c0-4adb-aaa5-84e208495de1 (eff89691-35c0-4adb-aaa5-84e208495de1): created successfully >2018-08-23 09:31:17.381 257256 INFO cinder.volume.manager [req-63f32b95-2863-4912-aab0-ba60eb254692 c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Created volume successfully. >2018-08-23 09:31:18.861 257031 INFO cinder.volume.flows.manager.create_volume [req-924c4c25-36eb-442f-9167-7e0a6f68521a c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Volume 1ad6f8e7-c6e6-4996-9ae7-5753cf49df24: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1ad6f8e7-c6e6-4996-9ae7-5753cf49df24'} >2018-08-23 09:31:19.478 257031 INFO cinder.volume.flows.manager.create_volume [req-924c4c25-36eb-442f-9167-7e0a6f68521a c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Volume volume-1ad6f8e7-c6e6-4996-9ae7-5753cf49df24 (1ad6f8e7-c6e6-4996-9ae7-5753cf49df24): created successfully >2018-08-23 09:31:19.482 257031 INFO cinder.volume.manager [req-924c4c25-36eb-442f-9167-7e0a6f68521a c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Created volume successfully. >2018-08-23 09:31:20.556 257256 INFO cinder.volume.flows.manager.create_volume [req-837a759e-4b66-42de-8cb9-282fe3d8fb1f cfa8290a82a14724add51012e3d92521 b4f8490f1e714ec8a4b746beed3af007 - default default] Volume 1f8ee235-5cda-494d-9f48-f41df8d345f6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1f8ee235-5cda-494d-9f48-f41df8d345f6'} >2018-08-23 09:31:21.293 257256 INFO cinder.volume.flows.manager.create_volume [req-837a759e-4b66-42de-8cb9-282fe3d8fb1f cfa8290a82a14724add51012e3d92521 b4f8490f1e714ec8a4b746beed3af007 - default default] Volume volume-1f8ee235-5cda-494d-9f48-f41df8d345f6 (1f8ee235-5cda-494d-9f48-f41df8d345f6): created successfully >2018-08-23 09:31:21.297 257256 INFO cinder.volume.manager [req-837a759e-4b66-42de-8cb9-282fe3d8fb1f cfa8290a82a14724add51012e3d92521 b4f8490f1e714ec8a4b746beed3af007 - default default] Created volume successfully. >2018-08-23 09:31:23.074 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 09:31:23.234 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 09:31:23.256 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 09:31:23.304 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 09:31:24.899 257256 INFO cinder.volume.manager [req-6400cfe8-622d-4b68-8e04-e9e098e97f88 cfa8290a82a14724add51012e3d92521 b4f8490f1e714ec8a4b746beed3af007 - default default] Deleted volume successfully. >2018-08-23 09:31:25.989 257256 INFO cinder.volume.manager [req-f1904bee-ee1b-4ada-9ea3-42a39e2dc772 c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Deleted volume successfully. >2018-08-23 09:31:26.009 257256 INFO cinder.volume.manager [req-3957a7a3-31a7-4640-8be4-a30c3945842c c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Deleted volume successfully. >2018-08-23 09:31:26.230 257031 INFO cinder.volume.manager [req-c9c6c579-8d8d-4cab-9a5d-2a413791e697 c36219c5f1c34b78a8921664472d1987 cc2a4b5a6b4446a4ac841f5550bb3778 - default default] Deleted volume successfully. >2018-08-23 09:31:32.469 257256 INFO cinder.volume.flows.manager.create_volume [req-bd4376af-6fe5-4ea9-ace3-4b81f608f1e7 4801c27bbc874b51bcb0ac9dd1b3eeec 8298cf4319e9419aba9d3e3123773451 - default default] Volume 09652498-4b92-49ac-9e02-1da8922feef0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-09652498-4b92-49ac-9e02-1da8922feef0'} >2018-08-23 09:31:35.771 257256 INFO cinder.volume.flows.manager.create_volume [req-bd4376af-6fe5-4ea9-ace3-4b81f608f1e7 4801c27bbc874b51bcb0ac9dd1b3eeec 8298cf4319e9419aba9d3e3123773451 - default default] Volume volume-09652498-4b92-49ac-9e02-1da8922feef0 (09652498-4b92-49ac-9e02-1da8922feef0): created successfully >2018-08-23 09:31:35.775 257256 INFO cinder.volume.manager [req-bd4376af-6fe5-4ea9-ace3-4b81f608f1e7 4801c27bbc874b51bcb0ac9dd1b3eeec 8298cf4319e9419aba9d3e3123773451 - default default] Created volume successfully. >2018-08-23 09:31:40.273 257256 INFO cinder.volume.manager [req-2848f2c3-4ca3-40f5-981b-3c1ea972e20a 4801c27bbc874b51bcb0ac9dd1b3eeec 8298cf4319e9419aba9d3e3123773451 - default default] Deleted volume successfully. >2018-08-23 09:31:47.356 257256 INFO cinder.volume.flows.manager.create_volume [req-2ea5ab10-b38f-49eb-b832-28b8151d1760 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] Volume fc5548d9-e4de-47ce-b70c-bc99d7435b0d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fc5548d9-e4de-47ce-b70c-bc99d7435b0d'} >2018-08-23 09:31:47.890 257256 INFO cinder.volume.flows.manager.create_volume [req-2ea5ab10-b38f-49eb-b832-28b8151d1760 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] Volume volume-fc5548d9-e4de-47ce-b70c-bc99d7435b0d (fc5548d9-e4de-47ce-b70c-bc99d7435b0d): created successfully >2018-08-23 09:31:47.895 257256 INFO cinder.volume.manager [req-2ea5ab10-b38f-49eb-b832-28b8151d1760 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] Created volume successfully. >2018-08-23 09:31:49.864 257256 INFO cinder.volume.manager [req-755a2ce1-73f1-4363-be5a-407787356cfa 32643de5d9794a48a062a144d6759d50 1b277169fc5f4064af046cc6f2dd808e - default default] Transfer volume completed successfully. >2018-08-23 09:31:51.876 257256 INFO cinder.volume.manager [req-fded7802-3819-4b7a-b4ef-2df757697265 2d13d9e4736a4002b1332af1f851c77b dfe2429b8da841be9ac21867b31b415f - default default] Deleted volume successfully. >2018-08-23 09:31:52.438 257256 INFO cinder.volume.flows.manager.create_volume [req-78ab2582-dede-4a7b-9902-5f85fb690f43 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] Volume 951e4682-8e2a-4e08-aed1-018ddf8afe21: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-951e4682-8e2a-4e08-aed1-018ddf8afe21'} >2018-08-23 09:31:53.034 257256 INFO cinder.volume.flows.manager.create_volume [req-78ab2582-dede-4a7b-9902-5f85fb690f43 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] Volume volume-951e4682-8e2a-4e08-aed1-018ddf8afe21 (951e4682-8e2a-4e08-aed1-018ddf8afe21): created successfully >2018-08-23 09:31:53.038 257256 INFO cinder.volume.manager [req-78ab2582-dede-4a7b-9902-5f85fb690f43 9025cf3c419c451aabb2499de6645b0b 68e3c0a6e1a6481da2fe169b09916bcd - default default] Created volume successfully. >2018-08-23 09:31:56.033 257256 INFO cinder.volume.manager [req-d9a49656-ba13-4edb-b585-8d4261a14943 2d13d9e4736a4002b1332af1f851c77b dfe2429b8da841be9ac21867b31b415f - default default] Deleted volume successfully. >2018-08-23 09:32:02.566 257256 INFO cinder.volume.flows.manager.create_volume [req-f66e28ca-fafa-4939-a80c-02eb446e750c 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Volume db4aeeda-8f51-4a44-8514-bc82d30c2e4c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-db4aeeda-8f51-4a44-8514-bc82d30c2e4c'} >2018-08-23 09:32:03.461 257256 INFO cinder.volume.flows.manager.create_volume [req-f66e28ca-fafa-4939-a80c-02eb446e750c 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Volume volume-db4aeeda-8f51-4a44-8514-bc82d30c2e4c (db4aeeda-8f51-4a44-8514-bc82d30c2e4c): created successfully >2018-08-23 09:32:03.465 257256 INFO cinder.volume.manager [req-f66e28ca-fafa-4939-a80c-02eb446e750c 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Created volume successfully. >2018-08-23 09:32:21.798 257256 INFO oslo.privsep.daemon [req-b3607392-a8a0-4c7f-a4d8-079077b2a65e 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpW8BZ9E/privsep.sock'] >2018-08-23 09:32:22.627 257256 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-23 09:32:22.733 257256 INFO oslo.privsep.daemon [req-b3607392-a8a0-4c7f-a4d8-079077b2a65e 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Spawned new privsep daemon via rootwrap >2018-08-23 09:32:22.670 322942 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-23 09:32:22.673 322942 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-23 09:32:22.676 322942 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-23 09:32:22.676 322942 INFO oslo.privsep.daemon [-] privsep daemon running as pid 322942 >2018-08-23 09:32:36.959 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 09:32:39.550 257256 WARNING oslo_config.cfg [req-b3607392-a8a0-4c7f-a4d8-079077b2a65e 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-23 09:32:55.411 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b3607392-a8a0-4c7f-a4d8-079077b2a65e 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 09:32:55.785 257256 INFO cinder.volume.manager [req-b3607392-a8a0-4c7f-a4d8-079077b2a65e 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Copy volume to image completed successfully. >2018-08-23 09:32:58.649 257256 INFO cinder.volume.manager [req-277806cb-391e-46c1-a2bb-c0b805bff014 301bec44b584407196380e1ab85fcc9e 9b9671386a4943758888c0803652bc10 - default default] Deleted volume successfully. >2018-08-23 09:33:04.693 257256 INFO cinder.volume.flows.manager.create_volume [req-1e8cc7c9-d784-45f4-9432-9adf52499d7b 422a8b3b1590441c8aa0837f52070b1d 1d6d29049a9341a096b840d8538187a0 - default default] Volume 7f1f3610-c659-446f-bcaa-c275449d8c4e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7f1f3610-c659-446f-bcaa-c275449d8c4e'} >2018-08-23 09:33:05.317 257256 INFO cinder.volume.flows.manager.create_volume [req-1e8cc7c9-d784-45f4-9432-9adf52499d7b 422a8b3b1590441c8aa0837f52070b1d 1d6d29049a9341a096b840d8538187a0 - default default] Volume volume-7f1f3610-c659-446f-bcaa-c275449d8c4e (7f1f3610-c659-446f-bcaa-c275449d8c4e): created successfully >2018-08-23 09:33:05.321 257256 INFO cinder.volume.manager [req-1e8cc7c9-d784-45f4-9432-9adf52499d7b 422a8b3b1590441c8aa0837f52070b1d 1d6d29049a9341a096b840d8538187a0 - default default] Created volume successfully. >2018-08-23 09:33:15.109 257256 INFO cinder.volume.manager [req-efd53b17-79d2-4e03-a32c-fd8231575d20 422a8b3b1590441c8aa0837f52070b1d 1d6d29049a9341a096b840d8538187a0 - default default] Deleted volume successfully. >2018-08-23 09:33:20.637 257256 INFO cinder.volume.flows.manager.create_volume [req-bfaef3dc-d0f0-4807-965d-db4ed94082d7 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] Volume 616eb26a-e076-41ac-b900-de7602324e01: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-616eb26a-e076-41ac-b900-de7602324e01'} >2018-08-23 09:33:21.165 257256 INFO cinder.volume.flows.manager.create_volume [req-bfaef3dc-d0f0-4807-965d-db4ed94082d7 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] Volume volume-616eb26a-e076-41ac-b900-de7602324e01 (616eb26a-e076-41ac-b900-de7602324e01): created successfully >2018-08-23 09:33:21.169 257256 INFO cinder.volume.manager [req-bfaef3dc-d0f0-4807-965d-db4ed94082d7 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] Created volume successfully. >2018-08-23 09:33:22.628 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-4bc1202b-46da-4a45-bf1f-4150f889f8dd 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] Unable to get remote copy information for volume volume-616eb26a-e076-41ac-b900-de7602324e01 >2018-08-23 09:33:22.920 257256 INFO cinder.volume.manager [req-4bc1202b-46da-4a45-bf1f-4150f889f8dd 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] Extend volume completed successfully. >2018-08-23 09:33:25.429 257256 INFO cinder.volume.manager [req-b7300b58-d915-42d7-bae9-bc7a58e4f176 937472f5162e4caca41f4e2da6a01b38 1e606a8c19ef4d8190419693061bbc63 - default default] Deleted volume successfully. >2018-08-23 09:33:30.751 257256 INFO cinder.volume.flows.manager.create_volume [req-0a9a2009-2b3b-4eeb-8e09-ea1cb72e13ae 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume 23463097-90ff-426e-bae7-ba95cc4b6ff1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-23463097-90ff-426e-bae7-ba95cc4b6ff1'} >2018-08-23 09:33:31.271 257256 INFO cinder.volume.flows.manager.create_volume [req-0a9a2009-2b3b-4eeb-8e09-ea1cb72e13ae 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume volume-23463097-90ff-426e-bae7-ba95cc4b6ff1 (23463097-90ff-426e-bae7-ba95cc4b6ff1): created successfully >2018-08-23 09:33:31.275 257256 INFO cinder.volume.manager [req-0a9a2009-2b3b-4eeb-8e09-ea1cb72e13ae 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Created volume successfully. >2018-08-23 09:33:33.675 257256 INFO cinder.volume.flows.manager.create_volume [req-3d3811ae-02b7-4014-b385-d4e50e78313d 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume 7f902a25-0085-4405-974a-9dae3ed1057d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7f902a25-0085-4405-974a-9dae3ed1057d'} >2018-08-23 09:33:34.198 257256 INFO cinder.volume.flows.manager.create_volume [req-3d3811ae-02b7-4014-b385-d4e50e78313d 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume volume-7f902a25-0085-4405-974a-9dae3ed1057d (7f902a25-0085-4405-974a-9dae3ed1057d): created successfully >2018-08-23 09:33:34.202 257256 INFO cinder.volume.manager [req-3d3811ae-02b7-4014-b385-d4e50e78313d 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Created volume successfully. >2018-08-23 09:33:36.387 257256 INFO cinder.volume.manager [req-acce49cb-e175-4adf-a0f0-af98ff4eeafa 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Deleted volume successfully. >2018-08-23 09:33:37.895 257256 INFO cinder.volume.manager [req-19cafeb9-2bb8-4489-a9aa-c45ca66ae0c1 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Deleted volume successfully. >2018-08-23 09:33:38.196 257256 INFO cinder.volume.flows.manager.create_volume [req-0f44e7c0-c9e8-44ab-b1ed-2a26c0c9ef01 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume c3dba3e2-5456-470d-98ff-2bac38afb870: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c3dba3e2-5456-470d-98ff-2bac38afb870'} >2018-08-23 09:33:38.810 257256 INFO cinder.volume.flows.manager.create_volume [req-0f44e7c0-c9e8-44ab-b1ed-2a26c0c9ef01 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume volume-c3dba3e2-5456-470d-98ff-2bac38afb870 (c3dba3e2-5456-470d-98ff-2bac38afb870): created successfully >2018-08-23 09:33:38.813 257256 INFO cinder.volume.manager [req-0f44e7c0-c9e8-44ab-b1ed-2a26c0c9ef01 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Created volume successfully. >2018-08-23 09:33:39.612 257256 INFO cinder.volume.flows.manager.create_volume [req-995ffe0f-ae9b-483e-9fbb-97f06219b8a4 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume 1b0bd336-d710-4a65-8d9c-51615a0b637d: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1b0bd336-d710-4a65-8d9c-51615a0b637d', 'source_volid': 'c3dba3e2-5456-470d-98ff-2bac38afb870', 'source_volstatus': u'available'} >2018-08-23 09:33:41.078 257256 INFO cinder.volume.flows.manager.create_volume [req-995ffe0f-ae9b-483e-9fbb-97f06219b8a4 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume volume-1b0bd336-d710-4a65-8d9c-51615a0b637d (1b0bd336-d710-4a65-8d9c-51615a0b637d): created successfully >2018-08-23 09:33:41.082 257256 INFO cinder.volume.manager [req-995ffe0f-ae9b-483e-9fbb-97f06219b8a4 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Created volume successfully. >2018-08-23 09:33:42.682 257256 INFO cinder.volume.flows.manager.create_volume [req-acef3a7a-b37c-49d3-bd0d-34e27605a175 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume a462fa34-6d4d-4e82-86a2-c7587014d1e9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a462fa34-6d4d-4e82-86a2-c7587014d1e9'} >2018-08-23 09:33:44.967 257256 INFO cinder.volume.flows.manager.create_volume [req-acef3a7a-b37c-49d3-bd0d-34e27605a175 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume volume-a462fa34-6d4d-4e82-86a2-c7587014d1e9 (a462fa34-6d4d-4e82-86a2-c7587014d1e9): created successfully >2018-08-23 09:33:44.971 257256 INFO cinder.volume.manager [req-acef3a7a-b37c-49d3-bd0d-34e27605a175 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Created volume successfully. >2018-08-23 09:33:47.143 257256 INFO cinder.volume.manager [req-b90c118e-1263-4b68-a419-0fb303be1793 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Deleted volume successfully. >2018-08-23 09:33:51.578 257256 INFO cinder.volume.manager [req-81c9ce9f-9f23-4091-8b66-0d6aa10c687c 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Deleted volume successfully. >2018-08-23 09:33:53.018 257256 INFO cinder.volume.flows.manager.create_volume [req-2b72d30f-78f0-4ff3-ac6f-bee3ff8f7e20 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume 2c9f93ba-088c-4f7e-96fa-f8f4b22fdfb1: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-2c9f93ba-088c-4f7e-96fa-f8f4b22fdfb1', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54655dd0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 09:33:53.984 257256 INFO cinder.image.image_utils [req-2b72d30f-78f0-4ff3-ac6f-bee3ff8f7e20 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-23 09:34:02.567 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 09:34:07.233 257256 INFO cinder.image.image_utils [req-2b72d30f-78f0-4ff3-ac6f-bee3ff8f7e20 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 09:34:09.085 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-2b72d30f-78f0-4ff3-ac6f-bee3ff8f7e20 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 09:34:09.445 257256 INFO cinder.volume.flows.manager.create_volume [req-2b72d30f-78f0-4ff3-ac6f-bee3ff8f7e20 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume volume-2c9f93ba-088c-4f7e-96fa-f8f4b22fdfb1 (2c9f93ba-088c-4f7e-96fa-f8f4b22fdfb1): created successfully >2018-08-23 09:34:09.449 257256 INFO cinder.volume.manager [req-2b72d30f-78f0-4ff3-ac6f-bee3ff8f7e20 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Created volume successfully. >2018-08-23 09:34:11.027 257031 INFO cinder.volume.flows.manager.create_volume [req-65075313-c95f-4e11-a8c5-252468768a06 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume 32438cf7-2458-41cd-931d-65044b667210: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-32438cf7-2458-41cd-931d-65044b667210'} >2018-08-23 09:34:11.703 257031 INFO cinder.volume.flows.manager.create_volume [req-65075313-c95f-4e11-a8c5-252468768a06 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Volume volume-32438cf7-2458-41cd-931d-65044b667210 (32438cf7-2458-41cd-931d-65044b667210): created successfully >2018-08-23 09:34:11.707 257031 INFO cinder.volume.manager [req-65075313-c95f-4e11-a8c5-252468768a06 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Created volume successfully. >2018-08-23 09:34:13.496 257031 INFO cinder.volume.manager [req-e4aa43cd-5f1c-4489-9344-47f746163453 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Deleted volume successfully. >2018-08-23 09:34:18.561 257256 INFO cinder.volume.manager [req-1c1c0650-b2b6-4775-9ac2-529c2d42855b 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Deleted volume successfully. >2018-08-23 09:34:19.990 257256 INFO cinder.volume.manager [req-98e4b702-956b-4629-a406-4077b8ccf27c 22402b1097f14d8c8a727e8305698045 e7c99ab1fc844a75a2fb490356759217 - default default] Deleted volume successfully. >2018-08-23 09:34:29.573 257256 INFO cinder.volume.flows.manager.create_volume [req-aff3b0db-3a7f-4eec-bd8a-6bf258c4c916 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Volume 8a2947ee-b6c0-41c9-b2e0-849bc688129c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8a2947ee-b6c0-41c9-b2e0-849bc688129c'} >2018-08-23 09:34:30.319 257256 INFO cinder.volume.flows.manager.create_volume [req-aff3b0db-3a7f-4eec-bd8a-6bf258c4c916 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Volume volume-8a2947ee-b6c0-41c9-b2e0-849bc688129c (8a2947ee-b6c0-41c9-b2e0-849bc688129c): created successfully >2018-08-23 09:34:30.323 257256 INFO cinder.volume.manager [req-aff3b0db-3a7f-4eec-bd8a-6bf258c4c916 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Created volume successfully. >2018-08-23 09:34:31.729 257256 INFO cinder.volume.flows.manager.create_volume [req-ad32be60-a422-403c-8d04-af1fb0ae9cb4 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Volume 67d6c805-47a4-47d8-9196-3a40ab4d0667: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-67d6c805-47a4-47d8-9196-3a40ab4d0667'} >2018-08-23 09:34:32.433 257256 INFO cinder.volume.flows.manager.create_volume [req-ad32be60-a422-403c-8d04-af1fb0ae9cb4 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Volume volume-67d6c805-47a4-47d8-9196-3a40ab4d0667 (67d6c805-47a4-47d8-9196-3a40ab4d0667): created successfully >2018-08-23 09:34:32.436 257256 INFO cinder.volume.manager [req-ad32be60-a422-403c-8d04-af1fb0ae9cb4 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Created volume successfully. >2018-08-23 09:34:33.803 257256 INFO cinder.volume.flows.manager.create_volume [req-19caa51e-0050-4c43-86eb-31f6103179dc 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Volume 56d160f0-cfc5-4e09-a59e-9f80aab356b9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-56d160f0-cfc5-4e09-a59e-9f80aab356b9'} >2018-08-23 09:34:34.338 257256 INFO cinder.volume.flows.manager.create_volume [req-19caa51e-0050-4c43-86eb-31f6103179dc 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Volume volume-56d160f0-cfc5-4e09-a59e-9f80aab356b9 (56d160f0-cfc5-4e09-a59e-9f80aab356b9): created successfully >2018-08-23 09:34:34.342 257256 INFO cinder.volume.manager [req-19caa51e-0050-4c43-86eb-31f6103179dc 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Created volume successfully. >2018-08-23 09:34:40.008 257256 INFO cinder.volume.manager [req-d6faade5-bec2-4247-98d0-43bdbe5aac22 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Deleted volume successfully. >2018-08-23 09:34:40.036 257256 INFO cinder.volume.manager [req-c32840c5-f2da-4a3c-9409-34c0636d0e15 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Deleted volume successfully. >2018-08-23 09:34:40.263 257256 INFO cinder.volume.manager [req-f42daaf2-d4a6-4acc-912e-bd842bfe81b8 3f2c15c51cc6440893082a647654b607 c9f09c0fc88d46e3b56b1acd1e998532 - default default] Deleted volume successfully. >2018-08-23 09:34:44.859 257256 INFO cinder.volume.flows.manager.create_volume [req-a0d99157-749b-4383-b292-086890e7f3f1 aa34ba5830f14cf6a7f473e635256fe0 ef4de57d7cee4d839626ad7d5c3e0b32 - default default] Volume 4e4374dd-d1f3-4ee2-ae38-9723d1962dbb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4e4374dd-d1f3-4ee2-ae38-9723d1962dbb'} >2018-08-23 09:34:45.384 257256 INFO cinder.volume.flows.manager.create_volume [req-a0d99157-749b-4383-b292-086890e7f3f1 aa34ba5830f14cf6a7f473e635256fe0 ef4de57d7cee4d839626ad7d5c3e0b32 - default default] Volume volume-4e4374dd-d1f3-4ee2-ae38-9723d1962dbb (4e4374dd-d1f3-4ee2-ae38-9723d1962dbb): created successfully >2018-08-23 09:34:45.388 257256 INFO cinder.volume.manager [req-a0d99157-749b-4383-b292-086890e7f3f1 aa34ba5830f14cf6a7f473e635256fe0 ef4de57d7cee4d839626ad7d5c3e0b32 - default default] Created volume successfully. >2018-08-23 09:35:06.378 257256 INFO cinder.volume.manager [req-5f149486-b3cf-4d71-9e7d-9095e1c803cd aa34ba5830f14cf6a7f473e635256fe0 ef4de57d7cee4d839626ad7d5c3e0b32 - default default] Deleted volume successfully. >2018-08-23 09:35:10.875 257256 INFO cinder.volume.flows.manager.create_volume [req-a3dcadbc-6b05-48f4-a37f-7e9e3eccd338 3594b0c5cf994e7c9566fc0fd4195d00 23d652262aac42f484fb43d92b259adf - default default] Volume 17806064-37f6-482d-8f8b-2a89e4aff5cc: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-17806064-37f6-482d-8f8b-2a89e4aff5cc'} >2018-08-23 09:35:11.424 257256 INFO cinder.volume.flows.manager.create_volume [req-a3dcadbc-6b05-48f4-a37f-7e9e3eccd338 3594b0c5cf994e7c9566fc0fd4195d00 23d652262aac42f484fb43d92b259adf - default default] Volume volume-17806064-37f6-482d-8f8b-2a89e4aff5cc (17806064-37f6-482d-8f8b-2a89e4aff5cc): created successfully >2018-08-23 09:35:11.428 257256 INFO cinder.volume.manager [req-a3dcadbc-6b05-48f4-a37f-7e9e3eccd338 3594b0c5cf994e7c9566fc0fd4195d00 23d652262aac42f484fb43d92b259adf - default default] Created volume successfully. >2018-08-23 09:35:14.124 257256 INFO cinder.volume.manager [req-a94621c3-870a-4a4c-a9b4-f4a2a1b7331e 3594b0c5cf994e7c9566fc0fd4195d00 23d652262aac42f484fb43d92b259adf - default default] Deleted volume successfully. >2018-08-23 09:35:57.926 257256 INFO cinder.volume.flows.manager.create_volume [req-b4bf5ed0-930e-4d17-9062-3978abeb8907 58f64149119a4eaab3f121df6d92e274 e3e8f16861c9417293a10ac47d098665 - default default] Volume 5f38c73b-1a7e-4c7f-8ccd-62caeaac90a7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5f38c73b-1a7e-4c7f-8ccd-62caeaac90a7'} >2018-08-23 09:35:58.469 257256 INFO cinder.volume.flows.manager.create_volume [req-b4bf5ed0-930e-4d17-9062-3978abeb8907 58f64149119a4eaab3f121df6d92e274 e3e8f16861c9417293a10ac47d098665 - default default] Volume volume-5f38c73b-1a7e-4c7f-8ccd-62caeaac90a7 (5f38c73b-1a7e-4c7f-8ccd-62caeaac90a7): created successfully >2018-08-23 09:35:58.472 257256 INFO cinder.volume.manager [req-b4bf5ed0-930e-4d17-9062-3978abeb8907 58f64149119a4eaab3f121df6d92e274 e3e8f16861c9417293a10ac47d098665 - default default] Created volume successfully. >2018-08-23 09:36:02.528 257256 INFO cinder.volume.manager [req-1bb8b89e-65ae-40a0-bf16-286bec7a575d 58f64149119a4eaab3f121df6d92e274 e3e8f16861c9417293a10ac47d098665 - default default] Deleted volume successfully. >2018-08-23 09:36:08.116 257256 INFO cinder.volume.flows.manager.create_volume [req-467fe6ac-f820-4258-8ff9-07cd9686b3d8 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Volume 330e8619-b494-40bb-b201-379032e784d7: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-330e8619-b494-40bb-b201-379032e784d7', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54c04590>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 09:36:09.132 257256 INFO cinder.image.image_utils [req-467fe6ac-f820-4258-8ff9-07cd9686b3d8 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Image download 12.00 MB at 11.83 MB/s >2018-08-23 09:36:27.188 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 09:36:31.951 257256 INFO cinder.image.image_utils [req-467fe6ac-f820-4258-8ff9-07cd9686b3d8 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 09:36:34.461 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-467fe6ac-f820-4258-8ff9-07cd9686b3d8 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 09:36:35.090 257256 INFO cinder.volume.flows.manager.create_volume [req-467fe6ac-f820-4258-8ff9-07cd9686b3d8 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Volume volume-330e8619-b494-40bb-b201-379032e784d7 (330e8619-b494-40bb-b201-379032e784d7): created successfully >2018-08-23 09:36:35.094 257256 INFO cinder.volume.manager [req-467fe6ac-f820-4258-8ff9-07cd9686b3d8 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Created volume successfully. >2018-08-23 09:36:36.588 257256 INFO cinder.volume.flows.manager.create_volume [req-43b471e6-a086-4b54-bb4e-f9826c105c2a 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Volume 0ae781b5-b1b9-45f8-b939-bf857290bd1e: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0ae781b5-b1b9-45f8-b939-bf857290bd1e', 'source_volid': '330e8619-b494-40bb-b201-379032e784d7', 'source_volstatus': u'available'} >2018-08-23 09:36:45.369 257256 INFO cinder.volume.flows.manager.create_volume [req-43b471e6-a086-4b54-bb4e-f9826c105c2a 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Volume volume-0ae781b5-b1b9-45f8-b939-bf857290bd1e (0ae781b5-b1b9-45f8-b939-bf857290bd1e): created successfully >2018-08-23 09:36:45.375 257256 INFO cinder.volume.manager [req-43b471e6-a086-4b54-bb4e-f9826c105c2a 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Created volume successfully. >2018-08-23 09:36:50.889 257256 INFO cinder.volume.manager [req-df88c379-0950-4ccf-9e8d-063f84771ce1 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Deleted volume successfully. >2018-08-23 09:36:51.784 257256 INFO cinder.volume.manager [req-0f1bbb8d-337e-43a7-8a18-cc96486e09e2 095e5e6e8b04406fa1ba93371f9b1876 e740b6b06dd0485ab553ecab6ce1a972 - default default] Deleted volume successfully. >2018-08-23 09:36:56.308 257256 INFO cinder.volume.flows.manager.create_volume [req-2238fe12-cc5a-4999-a26b-b983061322a6 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Volume c50349d2-0fed-492b-ba3b-4778dea08ed5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c50349d2-0fed-492b-ba3b-4778dea08ed5'} >2018-08-23 09:36:57.860 257256 INFO cinder.volume.flows.manager.create_volume [req-2238fe12-cc5a-4999-a26b-b983061322a6 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Volume volume-c50349d2-0fed-492b-ba3b-4778dea08ed5 (c50349d2-0fed-492b-ba3b-4778dea08ed5): created successfully >2018-08-23 09:36:57.864 257256 INFO cinder.volume.manager [req-2238fe12-cc5a-4999-a26b-b983061322a6 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Created volume successfully. >2018-08-23 09:37:00.054 257031 INFO cinder.volume.flows.manager.create_volume [req-25ad122d-796e-4a0c-b902-75f2d79133ca 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Volume 78eab6cd-d5c2-486f-a95f-6e262a1a011f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-78eab6cd-d5c2-486f-a95f-6e262a1a011f'} >2018-08-23 09:37:01.330 257031 INFO cinder.volume.flows.manager.create_volume [req-25ad122d-796e-4a0c-b902-75f2d79133ca 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Volume volume-78eab6cd-d5c2-486f-a95f-6e262a1a011f (78eab6cd-d5c2-486f-a95f-6e262a1a011f): created successfully >2018-08-23 09:37:01.334 257031 INFO cinder.volume.manager [req-25ad122d-796e-4a0c-b902-75f2d79133ca 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Created volume successfully. >2018-08-23 09:37:02.747 257256 INFO cinder.volume.flows.manager.create_volume [req-42dac544-713d-4f18-bde8-f63ceb1924b4 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Volume a6da9671-9d5d-43fe-ae3d-cd1985374c31: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a6da9671-9d5d-43fe-ae3d-cd1985374c31'} >2018-08-23 09:37:03.931 257256 INFO cinder.volume.flows.manager.create_volume [req-42dac544-713d-4f18-bde8-f63ceb1924b4 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Volume volume-a6da9671-9d5d-43fe-ae3d-cd1985374c31 (a6da9671-9d5d-43fe-ae3d-cd1985374c31): created successfully >2018-08-23 09:37:03.935 257256 INFO cinder.volume.manager [req-42dac544-713d-4f18-bde8-f63ceb1924b4 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Created volume successfully. >2018-08-23 09:37:08.879 257256 INFO cinder.volume.manager [req-01e3831e-edbb-41ad-a43b-88028773f648 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Deleted volume successfully. >2018-08-23 09:37:09.126 257031 INFO cinder.volume.manager [req-15d6ad99-9140-4718-a257-b98d85ed029b 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Deleted volume successfully. >2018-08-23 09:37:09.303 257256 INFO cinder.volume.manager [req-6f20ab30-23b6-4cdc-976d-3daa7c0e33d2 3ed0f3b0e40e4c98af0e578431ec2e42 3756cce6700b47279d9177560b1a7077 - default default] Deleted volume successfully. >2018-08-23 09:37:29.425 257031 INFO cinder.volume.flows.manager.create_volume [req-e571e0f6-709b-4462-b20e-18b7d6294469 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Volume 9f97fbb9-9012-4685-a436-cf18324eb4b9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9f97fbb9-9012-4685-a436-cf18324eb4b9'} >2018-08-23 09:37:29.726 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 09:37:29.796 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 09:37:30.058 257031 INFO cinder.volume.flows.manager.create_volume [req-e571e0f6-709b-4462-b20e-18b7d6294469 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Volume volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 (9f97fbb9-9012-4685-a436-cf18324eb4b9): created successfully >2018-08-23 09:37:30.061 257031 INFO cinder.volume.manager [req-e571e0f6-709b-4462-b20e-18b7d6294469 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Created volume successfully. >2018-08-23 09:37:31.719 257031 INFO cinder.rpc [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-23 09:37:31.724 257031 INFO cinder.rpc [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-23 09:37:31.867 257256 INFO cinder.volume.flows.manager.create_volume [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Volume ebc4648f-d4af-47af-bfc5-0e45cbaef7e2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2'} >2018-08-23 09:37:32.431 257256 INFO cinder.volume.flows.manager.create_volume [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Volume volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2 (ebc4648f-d4af-47af-bfc5-0e45cbaef7e2): created successfully >2018-08-23 09:37:32.435 257256 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Created volume successfully. >2018-08-23 09:37:32.863 257031 INFO oslo.privsep.daemon [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpeVGUCK/privsep.sock'] >2018-08-23 09:37:33.664 257031 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-23 09:37:33.764 257031 INFO oslo.privsep.daemon [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Spawned new privsep daemon via rootwrap >2018-08-23 09:37:33.707 352945 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-23 09:37:33.711 352945 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-23 09:37:33.713 352945 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-23 09:37:33.713 352945 INFO oslo.privsep.daemon [-] privsep daemon running as pid 352945 >2018-08-23 09:37:42.521 257256 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Initialize volume connection completed successfully. >2018-08-23 09:37:42.911 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 09:37:47.890 257031 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Initialize volume connection completed successfully. >2018-08-23 09:37:48.277 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 09:37:53.951 257031 INFO cinder.volume.utils [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Volume copy 1024.00 MB at 376.42 MB/s >2018-08-23 09:37:56.115 257256 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Terminate volume connection completed successfully. >2018-08-23 09:37:56.158 257256 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Remove volume export completed successfully. >2018-08-23 09:37:58.280 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 09:37:58.556 257031 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Terminate volume connection completed successfully. >2018-08-23 09:37:58.598 257031 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Remove volume export completed successfully. >2018-08-23 09:37:58.845 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 09:38:02.086 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Error running SSH command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 09:38:02.088 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 09:38:02.088 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Error running SSH command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 09:38:02.089 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-9f97fbb9-9012-4685-a436-cf18324eb4b9', '"volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-9f97fbb9-9012-4685-a436-cf18324eb4b9 "volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 09:38:02.090 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Unable to rename the logical volume for volume: 9f97fbb9-9012-4685-a436-cf18324eb4b9: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-9f97fbb9-9012-4685-a436-cf18324eb4b9', '"volume-ebc4648f-d4af-47af-bfc5-0e45cbaef7e2"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-23 09:38:02.229 257031 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Complete-Migrate volume completed successfully. >2018-08-23 09:38:02.230 257031 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Migrate volume completed successfully. >2018-08-23 09:38:02.730 257031 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Retype volume completed successfully. >2018-08-23 09:38:03.354 257031 INFO cinder.volume.manager [req-1fceaa64-4442-4052-a838-c9ba80a35450 a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Deleted volume successfully. >2018-08-23 09:38:05.510 257256 WARNING cinder.quota [req-688bbb10-d848-4ecb-b3df-da723096dacf a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1785661705 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1785661705, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:38:05.511 257256 WARNING cinder.quota [req-688bbb10-d848-4ecb-b3df-da723096dacf a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1785661705 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1785661705, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:38:05.766 257256 INFO cinder.volume.manager [req-688bbb10-d848-4ecb-b3df-da723096dacf a27e9d60d6e64a27a9323771260be788 3e01e146411348df89d453b4288c4e4f - default default] Deleted volume successfully. >2018-08-23 09:38:26.909 257256 INFO cinder.volume.flows.manager.create_volume [req-1cbe8fd9-ba3d-4716-b773-85e3acf1659c f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Volume db3b31ca-8b61-4f8c-8c6b-125e09ab1a7b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-db3b31ca-8b61-4f8c-8c6b-125e09ab1a7b'} >2018-08-23 09:38:27.437 257256 INFO cinder.volume.flows.manager.create_volume [req-1cbe8fd9-ba3d-4716-b773-85e3acf1659c f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Volume volume-db3b31ca-8b61-4f8c-8c6b-125e09ab1a7b (db3b31ca-8b61-4f8c-8c6b-125e09ab1a7b): created successfully >2018-08-23 09:38:27.441 257256 INFO cinder.volume.manager [req-1cbe8fd9-ba3d-4716-b773-85e3acf1659c f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Created volume successfully. >2018-08-23 09:38:35.577 257256 INFO cinder.volume.manager [req-74b60d18-5421-4b3e-8f35-bdf43c505cce f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Create snapshot completed successfully >2018-08-23 09:38:39.059 257256 INFO cinder.volume.manager [req-42c33a94-7973-4657-b60e-595ce7188cf7 f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Create snapshot completed successfully >2018-08-23 09:38:41.868 257256 INFO cinder.volume.manager [req-5bca4317-4a2c-4a48-9e55-1a37646141c2 0ef6c7da9bfd4c9fabf09c813cf1cb84 683a10df9bbd44e590c6d56c1d3740c5 - default default] Delete snapshot completed successfully. >2018-08-23 09:38:44.249 257256 INFO cinder.volume.manager [req-a0d0b449-d07c-4005-affb-1c86787b3422 f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Create snapshot completed successfully >2018-08-23 09:38:47.522 257256 INFO cinder.volume.manager [req-6e5e782b-55db-4cfc-8000-535efefb983e 0ef6c7da9bfd4c9fabf09c813cf1cb84 683a10df9bbd44e590c6d56c1d3740c5 - default default] Delete snapshot completed successfully. >2018-08-23 09:38:49.403 257256 INFO cinder.volume.manager [req-1c81b075-09b3-4e56-af06-47a06864d1e2 f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Create snapshot completed successfully >2018-08-23 09:38:52.204 257256 INFO cinder.volume.manager [req-8e93e614-bc62-4085-a34a-94cc1bc07cbd 0ef6c7da9bfd4c9fabf09c813cf1cb84 683a10df9bbd44e590c6d56c1d3740c5 - default default] Delete snapshot completed successfully. >2018-08-23 09:38:54.539 257256 INFO cinder.volume.manager [req-d6d84316-a648-47aa-a01b-47a0c92916f2 f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Create snapshot completed successfully >2018-08-23 09:38:57.342 257256 INFO cinder.volume.manager [req-5fc178d9-b138-4596-ac8a-5ae84cebc449 0ef6c7da9bfd4c9fabf09c813cf1cb84 683a10df9bbd44e590c6d56c1d3740c5 - default default] Delete snapshot completed successfully. >2018-08-23 09:39:00.612 257256 INFO cinder.volume.manager [req-2a64e1b3-0354-4ecf-941f-10c46600d54b f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Delete snapshot completed successfully. >2018-08-23 09:39:02.262 257256 INFO cinder.volume.manager [req-f96c82e9-4f92-4dbb-9ea2-8f3c44369f81 f5407c397bad4ca08f7c22997621dd4d 941aa0aa36d34d3d864ed105c4efbbcb - default default] Deleted volume successfully. >2018-08-23 09:39:07.673 257256 INFO cinder.volume.flows.manager.create_volume [req-3d628f82-8fec-4cb9-b0c9-7fa368792ac8 2bbeb5655cd244dc82b9c42536ceca89 a65913afe7a942639df7a517ec4d03bf - default default] Volume 8b332264-3900-433b-b0b9-a325c986aa74: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8b332264-3900-433b-b0b9-a325c986aa74'} >2018-08-23 09:39:08.218 257256 INFO cinder.volume.flows.manager.create_volume [req-3d628f82-8fec-4cb9-b0c9-7fa368792ac8 2bbeb5655cd244dc82b9c42536ceca89 a65913afe7a942639df7a517ec4d03bf - default default] Volume volume-8b332264-3900-433b-b0b9-a325c986aa74 (8b332264-3900-433b-b0b9-a325c986aa74): created successfully >2018-08-23 09:39:08.222 257256 INFO cinder.volume.manager [req-3d628f82-8fec-4cb9-b0c9-7fa368792ac8 2bbeb5655cd244dc82b9c42536ceca89 a65913afe7a942639df7a517ec4d03bf - default default] Created volume successfully. >2018-08-23 09:39:11.219 257256 INFO cinder.volume.manager [req-5840a8d3-fba2-4f32-9955-e60a18d76c36 2bbeb5655cd244dc82b9c42536ceca89 a65913afe7a942639df7a517ec4d03bf - default default] Create snapshot completed successfully >2018-08-23 09:39:15.708 257256 INFO cinder.volume.manager [req-e4ec5f3f-d476-4ed7-84ad-bbb304da6a66 2bbeb5655cd244dc82b9c42536ceca89 a65913afe7a942639df7a517ec4d03bf - default default] Delete snapshot completed successfully. >2018-08-23 09:39:17.459 257256 INFO cinder.volume.manager [req-3a51cc86-829d-41f4-a041-ad8ae4182c07 2bbeb5655cd244dc82b9c42536ceca89 a65913afe7a942639df7a517ec4d03bf - default default] Deleted volume successfully. >2018-08-23 09:39:22.504 257256 INFO cinder.volume.flows.manager.create_volume [req-cfd8a751-0748-4c76-a6b7-c62a612046dc 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Volume 293ea691-00ee-4d12-b8d5-06f03b4da750: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-293ea691-00ee-4d12-b8d5-06f03b4da750'} >2018-08-23 09:39:23.033 257256 INFO cinder.volume.flows.manager.create_volume [req-cfd8a751-0748-4c76-a6b7-c62a612046dc 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Volume volume-293ea691-00ee-4d12-b8d5-06f03b4da750 (293ea691-00ee-4d12-b8d5-06f03b4da750): created successfully >2018-08-23 09:39:23.037 257256 INFO cinder.volume.manager [req-cfd8a751-0748-4c76-a6b7-c62a612046dc 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Created volume successfully. >2018-08-23 09:39:25.929 257256 INFO cinder.volume.manager [req-7c5e829d-b3b3-4fb9-8eda-727b85af45c9 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Create snapshot completed successfully >2018-08-23 09:39:42.584 257256 INFO cinder.volume.manager [req-b5245909-9caa-4881-b59d-088bcdfb934b 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Initialize volume connection completed successfully. >2018-08-23 09:40:05.624 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b5245909-9caa-4881-b59d-088bcdfb934b 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 09:40:06.033 257256 INFO cinder.volume.manager [req-b5245909-9caa-4881-b59d-088bcdfb934b 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Terminate volume connection completed successfully. >2018-08-23 09:40:06.075 257256 INFO cinder.volume.manager [req-b5245909-9caa-4881-b59d-088bcdfb934b 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Remove volume export completed successfully. >2018-08-23 09:40:10.889 257256 INFO cinder.volume.manager [req-b5245909-9caa-4881-b59d-088bcdfb934b 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Deleted volume successfully. >2018-08-23 09:40:21.696 257256 INFO cinder.volume.manager [req-cce2a592-d137-4df8-ba11-ed1282975f39 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Create snapshot completed successfully >2018-08-23 09:40:25.713 257256 INFO cinder.volume.manager [req-8554ff1e-7ffc-4616-b52c-ffd930f881a7 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Delete snapshot completed successfully. >2018-08-23 09:40:29.798 257256 INFO cinder.volume.manager [req-68cb74b2-6680-4230-adcc-b0aadd9148be 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Create snapshot completed successfully >2018-08-23 09:40:35.887 257256 INFO cinder.volume.flows.manager.create_volume [req-b74155ba-9aeb-4662-b819-b408d32c81cf 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Volume 12b527c7-3104-4a5e-8269-c5bb1ac4f74d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-12b527c7-3104-4a5e-8269-c5bb1ac4f74d'} >2018-08-23 09:40:36.781 257256 INFO cinder.volume.flows.manager.create_volume [req-b74155ba-9aeb-4662-b819-b408d32c81cf 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Volume volume-12b527c7-3104-4a5e-8269-c5bb1ac4f74d (12b527c7-3104-4a5e-8269-c5bb1ac4f74d): created successfully >2018-08-23 09:40:36.785 257256 INFO cinder.volume.manager [req-b74155ba-9aeb-4662-b819-b408d32c81cf 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Created volume successfully. >2018-08-23 09:40:39.784 257256 INFO cinder.volume.manager [req-9deeebe8-36d0-4269-8666-8ef0ec7c3825 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Create snapshot completed successfully >2018-08-23 09:40:40.614 257256 INFO cinder.volume.flows.manager.create_volume [req-4ccad229-be24-4cad-bbee-a43b5234019e 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Volume d51fee08-ff24-4288-b1e4-66267448b436: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': 'a3f03408-c76c-452e-be54-72db41d8717a', 'volume_name': 'volume-d51fee08-ff24-4288-b1e4-66267448b436'} >2018-08-23 09:41:31.447 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-4ccad229-be24-4cad-bbee-a43b5234019e 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Unable to get remote copy information for volume volume-d51fee08-ff24-4288-b1e4-66267448b436 >2018-08-23 09:41:31.752 257256 INFO cinder.volume.flows.manager.create_volume [req-4ccad229-be24-4cad-bbee-a43b5234019e 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Volume volume-d51fee08-ff24-4288-b1e4-66267448b436 (d51fee08-ff24-4288-b1e4-66267448b436): created successfully >2018-08-23 09:41:31.757 257256 INFO cinder.volume.manager [req-4ccad229-be24-4cad-bbee-a43b5234019e 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Created volume successfully. >2018-08-23 09:41:34.052 257256 INFO cinder.volume.manager [req-9b775b95-9e3b-4a06-8afe-3540c69560e1 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Deleted volume successfully. >2018-08-23 09:41:36.644 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 09:41:36.738 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 09:41:38.823 257256 INFO cinder.volume.manager [req-93f5d11d-2d7f-4ec2-9046-b16c37ef55c9 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Delete snapshot completed successfully. >2018-08-23 09:41:39.089 257256 INFO cinder.volume.manager [req-0bb812e2-fab7-43e6-9b95-83a368409b47 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Delete snapshot completed successfully. >2018-08-23 09:41:39.314 257256 INFO cinder.volume.manager [req-b268a2cc-cca2-44eb-8b86-498ce8b1b667 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Delete snapshot completed successfully. >2018-08-23 09:41:44.971 257256 WARNING cinder.db.sqlalchemy.api [req-6bd5d8f1-7a64-41da-a2b6-e660b421da75 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-23 09:41:44.996 257256 INFO cinder.volume.manager [req-1787efa1-9479-4f65-a01a-febee65dce1f 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Deleted volume successfully. >2018-08-23 09:41:45.274 257256 INFO cinder.volume.manager [req-6bd5d8f1-7a64-41da-a2b6-e660b421da75 9903a9bbf4a641d3a5e8b434998199db 22dcb40c88ec49819c7fe9761e6812c6 - default default] Deleted volume successfully. >2018-08-23 09:41:50.695 257256 INFO cinder.volume.flows.manager.create_volume [req-dbdf1a1e-fe8c-4536-a543-db0d772a1476 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Volume ce83e3e4-ccef-4a84-bb31-ebdd14f98416: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ce83e3e4-ccef-4a84-bb31-ebdd14f98416'} >2018-08-23 09:41:51.218 257256 INFO cinder.volume.flows.manager.create_volume [req-dbdf1a1e-fe8c-4536-a543-db0d772a1476 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Volume volume-ce83e3e4-ccef-4a84-bb31-ebdd14f98416 (ce83e3e4-ccef-4a84-bb31-ebdd14f98416): created successfully >2018-08-23 09:41:51.222 257256 INFO cinder.volume.manager [req-dbdf1a1e-fe8c-4536-a543-db0d772a1476 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Created volume successfully. >2018-08-23 09:41:59.270 257256 INFO cinder.volume.manager [req-eebbeb67-9389-4593-80a7-7d707119710a a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Create snapshot completed successfully >2018-08-23 09:42:06.572 257256 INFO cinder.volume.manager [req-343f4401-445d-44a7-a0c9-98b6896b5bfa a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Create snapshot completed successfully >2018-08-23 09:42:13.946 257256 INFO cinder.volume.manager [req-a430e0e6-1744-496e-bd8e-95ed0d88b797 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Create snapshot completed successfully >2018-08-23 09:42:19.848 257256 INFO cinder.volume.manager [req-00724e95-7e79-46e1-b726-d77c3c4267f5 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Delete snapshot completed successfully. >2018-08-23 09:42:20.144 257256 INFO cinder.volume.manager [req-5af15020-b934-4d58-a3d0-902f8ee279f9 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Delete snapshot completed successfully. >2018-08-23 09:42:20.357 257256 INFO cinder.volume.manager [req-ba1711aa-a4ec-427e-bc4a-630d24e5071d a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Delete snapshot completed successfully. >2018-08-23 09:42:22.500 257256 INFO cinder.volume.manager [req-3f14c578-ca90-406c-87ef-81155c4b1d91 a3b41d3540204966baf5b0c4a015a77e c28e9ecb8f014b6b8570793fb8b34b15 - default default] Deleted volume successfully. >2018-08-23 09:42:28.120 257256 INFO cinder.volume.flows.manager.create_volume [req-bff0b29c-6c6d-454f-b0a9-4e366980306f 7d0fa6fd38fc46a8b3342f177da8771e 8e8ef02815c54f45b683e861d60e62e4 - default default] Volume 803c5c61-f5dd-455d-a672-b89d3460b4eb: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-803c5c61-f5dd-455d-a672-b89d3460b4eb'} >2018-08-23 09:42:28.649 257256 INFO cinder.volume.flows.manager.create_volume [req-bff0b29c-6c6d-454f-b0a9-4e366980306f 7d0fa6fd38fc46a8b3342f177da8771e 8e8ef02815c54f45b683e861d60e62e4 - default default] Volume volume-803c5c61-f5dd-455d-a672-b89d3460b4eb (803c5c61-f5dd-455d-a672-b89d3460b4eb): created successfully >2018-08-23 09:42:28.653 257256 INFO cinder.volume.manager [req-bff0b29c-6c6d-454f-b0a9-4e366980306f 7d0fa6fd38fc46a8b3342f177da8771e 8e8ef02815c54f45b683e861d60e62e4 - default default] Created volume successfully. >2018-08-23 09:42:36.649 257256 INFO cinder.volume.manager [req-b45c45da-d043-455a-be4f-b2edf116cca4 7d0fa6fd38fc46a8b3342f177da8771e 8e8ef02815c54f45b683e861d60e62e4 - default default] Create snapshot completed successfully >2018-08-23 09:42:39.958 257256 INFO cinder.volume.manager [req-f8e0ca05-a3c8-4c05-992f-670e2d82f1f5 7d0fa6fd38fc46a8b3342f177da8771e 8e8ef02815c54f45b683e861d60e62e4 - default default] Delete snapshot completed successfully. >2018-08-23 09:42:41.839 257256 INFO cinder.volume.manager [req-623a34ae-6752-4f12-9edc-3e92705b83b9 7d0fa6fd38fc46a8b3342f177da8771e 8e8ef02815c54f45b683e861d60e62e4 - default default] Deleted volume successfully. >2018-08-23 09:43:03.099 257256 INFO cinder.volume.flows.manager.create_volume [req-f0181ecf-8148-4019-b36c-d8d739b3c894 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume c2f31ee6-ce18-4288-aec7-9e4654367b2c: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-c2f31ee6-ce18-4288-aec7-9e4654367b2c', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab546bc450>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 09:43:03.923 257256 INFO cinder.image.image_utils [req-f0181ecf-8148-4019-b36c-d8d739b3c894 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-23 09:43:13.293 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 09:43:17.994 257256 INFO cinder.image.image_utils [req-f0181ecf-8148-4019-b36c-d8d739b3c894 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 09:43:19.810 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f0181ecf-8148-4019-b36c-d8d739b3c894 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 09:43:20.157 257256 INFO cinder.volume.flows.manager.create_volume [req-f0181ecf-8148-4019-b36c-d8d739b3c894 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume volume-c2f31ee6-ce18-4288-aec7-9e4654367b2c (c2f31ee6-ce18-4288-aec7-9e4654367b2c): created successfully >2018-08-23 09:43:20.161 257256 INFO cinder.volume.manager [req-f0181ecf-8148-4019-b36c-d8d739b3c894 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Created volume successfully. >2018-08-23 09:43:21.548 257256 INFO cinder.volume.flows.manager.create_volume [req-ec522f1a-5e96-474e-a499-620cbf59384e f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume 4c17ad98-78de-4412-bab8-81309c246068: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4c17ad98-78de-4412-bab8-81309c246068', 'source_volid': 'c2f31ee6-ce18-4288-aec7-9e4654367b2c', 'source_volstatus': u'available'} >2018-08-23 09:43:28.212 257256 INFO cinder.volume.flows.manager.create_volume [req-ec522f1a-5e96-474e-a499-620cbf59384e f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume volume-4c17ad98-78de-4412-bab8-81309c246068 (4c17ad98-78de-4412-bab8-81309c246068): created successfully >2018-08-23 09:43:28.216 257256 INFO cinder.volume.manager [req-ec522f1a-5e96-474e-a499-620cbf59384e f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Created volume successfully. >2018-08-23 09:43:28.924 257256 INFO cinder.volume.flows.manager.create_volume [req-b37aa175-fea9-4bdc-b6c8-04db0d1f2d03 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume 1b95dcee-15c1-423c-bee7-f10822c491e2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1b95dcee-15c1-423c-bee7-f10822c491e2'} >2018-08-23 09:43:29.444 257256 INFO cinder.volume.flows.manager.create_volume [req-b37aa175-fea9-4bdc-b6c8-04db0d1f2d03 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume volume-1b95dcee-15c1-423c-bee7-f10822c491e2 (1b95dcee-15c1-423c-bee7-f10822c491e2): created successfully >2018-08-23 09:43:29.448 257256 INFO cinder.volume.manager [req-b37aa175-fea9-4bdc-b6c8-04db0d1f2d03 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Created volume successfully. >2018-08-23 09:43:30.312 257256 INFO cinder.volume.flows.manager.create_volume [req-755d4e68-06a4-4f90-a076-0730a5eef0d8 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume a3750935-ed4a-490c-8bb8-9aa3b529ad9e: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-a3750935-ed4a-490c-8bb8-9aa3b529ad9e', 'source_volid': '1b95dcee-15c1-423c-bee7-f10822c491e2', 'source_volstatus': u'available'} >2018-08-23 09:44:32.581 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-755d4e68-06a4-4f90-a076-0730a5eef0d8 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Unable to get remote copy information for volume volume-a3750935-ed4a-490c-8bb8-9aa3b529ad9e >2018-08-23 09:44:33.041 257256 INFO cinder.volume.flows.manager.create_volume [req-755d4e68-06a4-4f90-a076-0730a5eef0d8 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Volume volume-a3750935-ed4a-490c-8bb8-9aa3b529ad9e (a3750935-ed4a-490c-8bb8-9aa3b529ad9e): created successfully >2018-08-23 09:44:33.046 257256 INFO cinder.volume.manager [req-755d4e68-06a4-4f90-a076-0730a5eef0d8 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Created volume successfully. >2018-08-23 09:44:36.697 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-23 09:44:36.819 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-23 09:44:38.396 257256 INFO cinder.volume.manager [req-829c1f67-e799-40a6-a177-a5caa6f99f0f f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Deleted volume successfully. >2018-08-23 09:44:38.691 257256 INFO cinder.volume.manager [req-933e6dd4-14bc-4de6-9ae7-cba78d5cc0a8 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Deleted volume successfully. >2018-08-23 09:44:38.864 257256 INFO cinder.volume.manager [req-c3d38d6f-08b2-4eff-8c7f-5a97f6378017 f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Deleted volume successfully. >2018-08-23 09:44:38.977 257256 INFO cinder.volume.manager [req-432dcbe8-630d-4c56-af98-abefaf0d907e f91e6b4e4b4a49df828c4997d6ac2132 6b980778842a4fe1bd1339be0c0b984c - default default] Deleted volume successfully. >2018-08-23 09:44:43.302 257256 INFO cinder.volume.flows.manager.create_volume [req-5a30ab16-1a39-4f9f-bcfd-4389ee85ff59 1031a1baf23b44019713fdcc6a400519 bbb70627d6c547bda4167ce2a4ecc087 - default default] Volume b583c2db-fe22-4b65-9fd7-d1c1e27a1769: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-b583c2db-fe22-4b65-9fd7-d1c1e27a1769'} >2018-08-23 09:44:44.189 257256 INFO cinder.volume.flows.manager.create_volume [req-5a30ab16-1a39-4f9f-bcfd-4389ee85ff59 1031a1baf23b44019713fdcc6a400519 bbb70627d6c547bda4167ce2a4ecc087 - default default] Volume volume-b583c2db-fe22-4b65-9fd7-d1c1e27a1769 (b583c2db-fe22-4b65-9fd7-d1c1e27a1769): created successfully >2018-08-23 09:44:44.193 257256 INFO cinder.volume.manager [req-5a30ab16-1a39-4f9f-bcfd-4389ee85ff59 1031a1baf23b44019713fdcc6a400519 bbb70627d6c547bda4167ce2a4ecc087 - default default] Created volume successfully. >2018-08-23 09:44:47.243 257256 INFO cinder.volume.manager [req-65d2d80c-1752-4d68-a9cb-f3228a4b8c67 1031a1baf23b44019713fdcc6a400519 bbb70627d6c547bda4167ce2a4ecc087 - default default] Deleted volume successfully. >2018-08-23 09:45:12.487 257256 INFO cinder.volume.flows.manager.create_volume [req-7a290c61-2c88-4e7b-801c-ea3cb7b7b968 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] Volume 32f1bcf8-4212-4041-991c-4ff8d29e5532: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-32f1bcf8-4212-4041-991c-4ff8d29e5532'} >2018-08-23 09:45:13.012 257256 INFO cinder.volume.flows.manager.create_volume [req-7a290c61-2c88-4e7b-801c-ea3cb7b7b968 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] Volume volume-32f1bcf8-4212-4041-991c-4ff8d29e5532 (32f1bcf8-4212-4041-991c-4ff8d29e5532): created successfully >2018-08-23 09:45:13.016 257256 INFO cinder.volume.manager [req-7a290c61-2c88-4e7b-801c-ea3cb7b7b968 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] Created volume successfully. >2018-08-23 09:45:15.513 257256 INFO cinder.volume.manager [req-07de5268-c66b-4a8c-a502-d0a395f5746f be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] Deleted volume successfully. >2018-08-23 09:45:16.014 257256 INFO cinder.volume.flows.manager.create_volume [req-e42056b7-0c6d-497a-af76-c1371daab2b2 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] Volume 7ab19427-64ed-4be1-aedb-e21777c35afe: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7ab19427-64ed-4be1-aedb-e21777c35afe'} >2018-08-23 09:45:16.582 257256 INFO cinder.volume.flows.manager.create_volume [req-e42056b7-0c6d-497a-af76-c1371daab2b2 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] Volume volume-7ab19427-64ed-4be1-aedb-e21777c35afe (7ab19427-64ed-4be1-aedb-e21777c35afe): created successfully >2018-08-23 09:45:16.586 257256 INFO cinder.volume.manager [req-e42056b7-0c6d-497a-af76-c1371daab2b2 be2dc1c74bf44602b49e9c8baa55070a b62ec93da9334127badc96668cbda691 - default default] Created volume successfully. >2018-08-23 09:45:18.671 257256 INFO cinder.volume.manager [req-42b856a0-c171-47be-8935-6c87298c23f5 eac08123962c4aa7b739db066ee84e43 fb63ad21d47a4dd69dd4799f44f6c285 - default default] Transfer volume completed successfully. >2018-08-23 09:45:21.398 257256 INFO cinder.volume.manager [req-8e038666-502b-4eb4-849c-66417f440ebe d5c76c1c9de1419694293ac9318b2444 356659e3c173414396f1a931a8449d4f - default default] Deleted volume successfully. >2018-08-23 09:45:30.972 257256 INFO cinder.volume.flows.manager.create_volume [req-41d98c99-7ba3-4c8b-a68c-0a044c4a2358 c787ac949cda400f8cb79aff3f7a98dc 988012b9181e4f4ebf0fce87b87813f9 - default default] Volume 8839dfb3-ac19-4284-bdb0-911b95765c5f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8839dfb3-ac19-4284-bdb0-911b95765c5f'} >2018-08-23 09:45:31.519 257256 INFO cinder.volume.flows.manager.create_volume [req-41d98c99-7ba3-4c8b-a68c-0a044c4a2358 c787ac949cda400f8cb79aff3f7a98dc 988012b9181e4f4ebf0fce87b87813f9 - default default] Volume volume-8839dfb3-ac19-4284-bdb0-911b95765c5f (8839dfb3-ac19-4284-bdb0-911b95765c5f): created successfully >2018-08-23 09:45:31.523 257256 INFO cinder.volume.manager [req-41d98c99-7ba3-4c8b-a68c-0a044c4a2358 c787ac949cda400f8cb79aff3f7a98dc 988012b9181e4f4ebf0fce87b87813f9 - default default] Created volume successfully. >2018-08-23 09:45:34.911 257256 INFO cinder.volume.manager [req-f6172c45-217d-4a89-b8e1-d3d531f25fd0 c787ac949cda400f8cb79aff3f7a98dc 988012b9181e4f4ebf0fce87b87813f9 - default default] Deleted volume successfully. >2018-08-23 09:45:42.583 257256 INFO cinder.volume.flows.manager.create_volume [req-cd9ef6c5-b80d-4c01-af45-20c6cc0ae3a4 2ec230d16cac47adbe2ed4538cf0a029 4471e16e6c774231af205991fd3f03fc - default default] Volume 2caf3fc2-5820-42bc-b0d1-093741fd23aa: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2caf3fc2-5820-42bc-b0d1-093741fd23aa'} >2018-08-23 09:45:43.107 257256 INFO cinder.volume.flows.manager.create_volume [req-cd9ef6c5-b80d-4c01-af45-20c6cc0ae3a4 2ec230d16cac47adbe2ed4538cf0a029 4471e16e6c774231af205991fd3f03fc - default default] Volume volume-2caf3fc2-5820-42bc-b0d1-093741fd23aa (2caf3fc2-5820-42bc-b0d1-093741fd23aa): created successfully >2018-08-23 09:45:43.111 257256 INFO cinder.volume.manager [req-cd9ef6c5-b80d-4c01-af45-20c6cc0ae3a4 2ec230d16cac47adbe2ed4538cf0a029 4471e16e6c774231af205991fd3f03fc - default default] Created volume successfully. >2018-08-23 09:45:51.190 257256 INFO cinder.volume.manager [req-9c9ef482-bcc6-4850-ad49-182efb07e6c1 2ec230d16cac47adbe2ed4538cf0a029 4471e16e6c774231af205991fd3f03fc - default default] Create snapshot completed successfully >2018-08-23 09:45:56.861 257256 INFO cinder.volume.manager [req-19d3f633-4163-4ec8-b748-50ba8b3659aa 2ec230d16cac47adbe2ed4538cf0a029 4471e16e6c774231af205991fd3f03fc - default default] Delete snapshot completed successfully. >2018-08-23 09:45:58.691 257256 INFO cinder.volume.manager [req-de281640-e620-4ab1-a11c-f7cacbe3f68b 2ec230d16cac47adbe2ed4538cf0a029 4471e16e6c774231af205991fd3f03fc - default default] Deleted volume successfully. >2018-08-23 09:54:17.210 257256 INFO cinder.volume.flows.manager.create_volume [req-59aefbd0-3439-4090-8396-74737246b78e 1e51862745ed43fc93c7eddb7cb2ce87 97befaf4767a4de491a198add859188c - default default] Volume cadc15f8-3f26-4e00-8f44-64b73981fb6c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cadc15f8-3f26-4e00-8f44-64b73981fb6c'} >2018-08-23 09:54:17.745 257256 INFO cinder.volume.flows.manager.create_volume [req-59aefbd0-3439-4090-8396-74737246b78e 1e51862745ed43fc93c7eddb7cb2ce87 97befaf4767a4de491a198add859188c - default default] Volume volume-cadc15f8-3f26-4e00-8f44-64b73981fb6c (cadc15f8-3f26-4e00-8f44-64b73981fb6c): created successfully >2018-08-23 09:54:17.749 257256 INFO cinder.volume.manager [req-59aefbd0-3439-4090-8396-74737246b78e 1e51862745ed43fc93c7eddb7cb2ce87 97befaf4767a4de491a198add859188c - default default] Created volume successfully. >2018-08-23 09:54:20.538 257256 INFO cinder.volume.manager [req-1f89135d-bf25-423e-9dc4-56b0cd20fdfd 1e51862745ed43fc93c7eddb7cb2ce87 97befaf4767a4de491a198add859188c - default default] Deleted volume successfully. >2018-08-23 09:54:30.118 257256 INFO cinder.volume.flows.manager.create_volume [req-655bc067-c2d3-4449-863f-2135cf35243d 11a74f4df984439dad085f4571cede5f f49f2954dc0c4d1c986abbaa21a01f20 - default default] Volume 207891c0-8196-4eb9-9621-db2126884ff8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-207891c0-8196-4eb9-9621-db2126884ff8'} >2018-08-23 09:54:30.669 257256 INFO cinder.volume.flows.manager.create_volume [req-655bc067-c2d3-4449-863f-2135cf35243d 11a74f4df984439dad085f4571cede5f f49f2954dc0c4d1c986abbaa21a01f20 - default default] Volume volume-207891c0-8196-4eb9-9621-db2126884ff8 (207891c0-8196-4eb9-9621-db2126884ff8): created successfully >2018-08-23 09:54:30.672 257256 INFO cinder.volume.manager [req-655bc067-c2d3-4449-863f-2135cf35243d 11a74f4df984439dad085f4571cede5f f49f2954dc0c4d1c986abbaa21a01f20 - default default] Created volume successfully. >2018-08-23 09:54:34.495 257256 WARNING cinder.quota [req-61f20660-11ab-4883-820a-2bb0fbc31a26 11a74f4df984439dad085f4571cede5f f49f2954dc0c4d1c986abbaa21a01f20 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-160557367 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-160557367, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:54:34.496 257256 WARNING cinder.quota [req-61f20660-11ab-4883-820a-2bb0fbc31a26 11a74f4df984439dad085f4571cede5f f49f2954dc0c4d1c986abbaa21a01f20 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-160557367 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-160557367, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:54:34.762 257256 INFO cinder.volume.manager [req-61f20660-11ab-4883-820a-2bb0fbc31a26 11a74f4df984439dad085f4571cede5f f49f2954dc0c4d1c986abbaa21a01f20 - default default] Deleted volume successfully. >2018-08-23 09:54:44.282 257256 INFO cinder.volume.flows.manager.create_volume [req-24d9099b-8dcd-4503-a16e-6cd133870ac8 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] Volume be3a30ae-467a-466d-9cc2-54f20807bf8d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-be3a30ae-467a-466d-9cc2-54f20807bf8d'} >2018-08-23 09:54:44.899 257256 INFO cinder.volume.flows.manager.create_volume [req-24d9099b-8dcd-4503-a16e-6cd133870ac8 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] Volume volume-be3a30ae-467a-466d-9cc2-54f20807bf8d (be3a30ae-467a-466d-9cc2-54f20807bf8d): created successfully >2018-08-23 09:54:44.902 257256 INFO cinder.volume.manager [req-24d9099b-8dcd-4503-a16e-6cd133870ac8 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] Created volume successfully. >2018-08-23 09:54:46.558 257256 INFO cinder.volume.manager [req-0244e037-000f-47a1-8d1e-a17f98f6ac60 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] Retype volume completed successfully. >2018-08-23 09:54:50.430 257256 WARNING cinder.quota [req-be338d2c-389a-4750-b144-d4ae8960dff3 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1265939853 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1265939853, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:54:50.431 257256 WARNING cinder.quota [req-be338d2c-389a-4750-b144-d4ae8960dff3 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1265939853 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1265939853, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 09:54:50.680 257256 INFO cinder.volume.manager [req-be338d2c-389a-4750-b144-d4ae8960dff3 9e46104e94614e84bfabb3ce75a78348 f2ae8ab0d51e45b4b1b1002ee63e8e90 - default default] Deleted volume successfully. >2018-08-23 09:55:31.295 257256 INFO cinder.volume.flows.manager.create_volume [req-76bfc1d0-f29e-42b2-84b9-771f0916311b 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume 37a523c7-de47-41e7-ba1d-0097dd9d5f3e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-37a523c7-de47-41e7-ba1d-0097dd9d5f3e'} >2018-08-23 09:55:32.188 257256 INFO cinder.volume.flows.manager.create_volume [req-76bfc1d0-f29e-42b2-84b9-771f0916311b 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume volume-37a523c7-de47-41e7-ba1d-0097dd9d5f3e (37a523c7-de47-41e7-ba1d-0097dd9d5f3e): created successfully >2018-08-23 09:55:32.191 257256 INFO cinder.volume.manager [req-76bfc1d0-f29e-42b2-84b9-771f0916311b 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Created volume successfully. >2018-08-23 09:55:33.047 257256 INFO cinder.volume.manager [req-94ac24c2-e20f-43d9-bf1e-684892c38b86 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Attaching volume 37a523c7-de47-41e7-ba1d-0097dd9d5f3e to instance 34b12148-4859-4cad-8f81-86529a93df53 at mountpoint /dev/vdb on host None. >2018-08-23 09:55:33.125 257256 INFO cinder.volume.manager [req-94ac24c2-e20f-43d9-bf1e-684892c38b86 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Attach volume completed successfully. >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager [req-bd37af5f-1cdb-464b-b7a9-2a9a4e27baf5 b6dd6d5a0d164b0da0a6b051b109b469 0431c1a138a746b1b36dad4c76598628 - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager force=force) >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-23 09:55:34.022 257256 ERROR cinder.volume.manager >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server [req-bd37af5f-1cdb-464b-b7a9-2a9a4e27baf5 b6dd6d5a0d164b0da0a6b051b109b469 0431c1a138a746b1b36dad4c76598628 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-23 09:55:34.025 257256 ERROR oslo_messaging.rpc.server >2018-08-23 09:55:34.366 257256 INFO cinder.volume.manager [req-3dc95138-1b64-42c8-bdd6-088d02c60de2 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Detaching volume 37a523c7-de47-41e7-ba1d-0097dd9d5f3e from instance 34b12148-4859-4cad-8f81-86529a93df53. >2018-08-23 09:55:34.451 257256 INFO cinder.volume.manager [req-3dc95138-1b64-42c8-bdd6-088d02c60de2 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Detach volume completed successfully. >2018-08-23 09:55:38.061 257256 INFO cinder.volume.flows.manager.create_volume [req-92bd2700-e7b5-47c5-acf2-bb747ce71603 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume 79426e87-bf5b-4eb7-9bdb-1ca1f74bf5e6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-79426e87-bf5b-4eb7-9bdb-1ca1f74bf5e6'} >2018-08-23 09:55:38.584 257256 INFO cinder.volume.flows.manager.create_volume [req-92bd2700-e7b5-47c5-acf2-bb747ce71603 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume volume-79426e87-bf5b-4eb7-9bdb-1ca1f74bf5e6 (79426e87-bf5b-4eb7-9bdb-1ca1f74bf5e6): created successfully >2018-08-23 09:55:38.588 257256 INFO cinder.volume.manager [req-92bd2700-e7b5-47c5-acf2-bb747ce71603 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Created volume successfully. >2018-08-23 09:55:40.905 257256 INFO cinder.volume.manager [req-56a8abf0-ec99-4336-9824-8b467db8c7e8 b6dd6d5a0d164b0da0a6b051b109b469 0431c1a138a746b1b36dad4c76598628 - default default] Deleted volume successfully. >2018-08-23 09:55:41.701 257031 INFO cinder.volume.flows.manager.create_volume [req-e5ae610d-c360-4a22-9861-8c4a9f3f1501 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume e9cfec02-1838-4359-b2a6-e414533a666a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e9cfec02-1838-4359-b2a6-e414533a666a'} >2018-08-23 09:55:42.234 257031 INFO cinder.volume.flows.manager.create_volume [req-e5ae610d-c360-4a22-9861-8c4a9f3f1501 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume volume-e9cfec02-1838-4359-b2a6-e414533a666a (e9cfec02-1838-4359-b2a6-e414533a666a): created successfully >2018-08-23 09:55:42.238 257031 INFO cinder.volume.manager [req-e5ae610d-c360-4a22-9861-8c4a9f3f1501 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Created volume successfully. >2018-08-23 09:55:44.675 257031 INFO cinder.volume.manager [req-3d0bc6e9-4a29-4de4-98fd-b5bc3013bf37 b6dd6d5a0d164b0da0a6b051b109b469 0431c1a138a746b1b36dad4c76598628 - default default] Deleted volume successfully. >2018-08-23 09:55:45.385 257256 INFO cinder.volume.flows.manager.create_volume [req-d99869cc-ebe8-4012-bd44-4887a8b958b2 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume 8af33319-1180-4db0-8d82-191cd17ab92e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8af33319-1180-4db0-8d82-191cd17ab92e'} >2018-08-23 09:55:45.926 257256 INFO cinder.volume.flows.manager.create_volume [req-d99869cc-ebe8-4012-bd44-4887a8b958b2 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume volume-8af33319-1180-4db0-8d82-191cd17ab92e (8af33319-1180-4db0-8d82-191cd17ab92e): created successfully >2018-08-23 09:55:45.930 257256 INFO cinder.volume.manager [req-d99869cc-ebe8-4012-bd44-4887a8b958b2 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Created volume successfully. >2018-08-23 09:55:48.081 257256 INFO cinder.volume.manager [req-15d27786-281b-49e4-b0be-11db1ba45e2d b6dd6d5a0d164b0da0a6b051b109b469 0431c1a138a746b1b36dad4c76598628 - default default] Deleted volume successfully. >2018-08-23 09:55:48.826 257256 INFO cinder.volume.flows.manager.create_volume [req-34283f09-c041-48c9-84b4-c754536d8d93 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume c21ddedc-ca74-4ec2-8a4e-42ec1876033d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c21ddedc-ca74-4ec2-8a4e-42ec1876033d'} >2018-08-23 09:55:49.451 257256 INFO cinder.volume.flows.manager.create_volume [req-34283f09-c041-48c9-84b4-c754536d8d93 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume volume-c21ddedc-ca74-4ec2-8a4e-42ec1876033d (c21ddedc-ca74-4ec2-8a4e-42ec1876033d): created successfully >2018-08-23 09:55:49.455 257256 INFO cinder.volume.manager [req-34283f09-c041-48c9-84b4-c754536d8d93 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Created volume successfully. >2018-08-23 09:55:51.260 257256 INFO cinder.volume.manager [req-819c525e-7de8-49bb-8ed0-a193f3fd9aac b6dd6d5a0d164b0da0a6b051b109b469 0431c1a138a746b1b36dad4c76598628 - default default] Deleted volume successfully. >2018-08-23 09:55:51.604 257256 INFO cinder.volume.flows.manager.create_volume [req-3951c6a5-d8b8-4797-bc5f-a9e035538b2e 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume ce2912d0-4b33-415a-b639-b2cca1040f0b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ce2912d0-4b33-415a-b639-b2cca1040f0b'} >2018-08-23 09:55:52.287 257256 INFO cinder.volume.flows.manager.create_volume [req-3951c6a5-d8b8-4797-bc5f-a9e035538b2e 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Volume volume-ce2912d0-4b33-415a-b639-b2cca1040f0b (ce2912d0-4b33-415a-b639-b2cca1040f0b): created successfully >2018-08-23 09:55:52.291 257256 INFO cinder.volume.manager [req-3951c6a5-d8b8-4797-bc5f-a9e035538b2e 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Created volume successfully. >2018-08-23 09:55:55.198 257256 INFO cinder.volume.manager [req-56e233f6-26f0-40e0-a58d-34c6bfc252ef 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Deleted volume successfully. >2018-08-23 09:55:55.436 257256 INFO cinder.volume.manager [req-ec47d4b6-221d-43ca-8842-18c88358f5de 7d9b04b0ba8d4cf0813e3533484edd50 3d1c20c0092546499136407202be2dbf - default default] Deleted volume successfully. >2018-08-23 09:56:04.125 257256 INFO cinder.volume.flows.manager.create_volume [req-d8a9c0eb-4daa-4227-96da-a52032fbcf55 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Volume 74098009-775e-4241-8a37-564622a2b8c0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-74098009-775e-4241-8a37-564622a2b8c0'} >2018-08-23 09:56:04.647 257256 INFO cinder.volume.flows.manager.create_volume [req-d8a9c0eb-4daa-4227-96da-a52032fbcf55 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Volume volume-74098009-775e-4241-8a37-564622a2b8c0 (74098009-775e-4241-8a37-564622a2b8c0): created successfully >2018-08-23 09:56:04.651 257256 INFO cinder.volume.manager [req-d8a9c0eb-4daa-4227-96da-a52032fbcf55 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Created volume successfully. >2018-08-23 09:56:06.140 257256 INFO cinder.volume.flows.manager.create_volume [req-b6bf22a4-1f65-48b6-86a0-3d5b1e8ad209 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Volume 0af3d6b0-6dfc-4185-bd4d-ae72437fc551: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0af3d6b0-6dfc-4185-bd4d-ae72437fc551'} >2018-08-23 09:56:06.672 257256 INFO cinder.volume.flows.manager.create_volume [req-b6bf22a4-1f65-48b6-86a0-3d5b1e8ad209 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Volume volume-0af3d6b0-6dfc-4185-bd4d-ae72437fc551 (0af3d6b0-6dfc-4185-bd4d-ae72437fc551): created successfully >2018-08-23 09:56:06.676 257256 INFO cinder.volume.manager [req-b6bf22a4-1f65-48b6-86a0-3d5b1e8ad209 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Created volume successfully. >2018-08-23 09:56:08.246 257031 INFO cinder.volume.flows.manager.create_volume [req-3b16bf94-cbd6-4d3e-b4b3-6a6da55e994a 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Volume 48bc0c0d-95a6-4472-a96f-c1028aa33601: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-48bc0c0d-95a6-4472-a96f-c1028aa33601'} >2018-08-23 09:56:08.773 257031 INFO cinder.volume.flows.manager.create_volume [req-3b16bf94-cbd6-4d3e-b4b3-6a6da55e994a 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Volume volume-48bc0c0d-95a6-4472-a96f-c1028aa33601 (48bc0c0d-95a6-4472-a96f-c1028aa33601): created successfully >2018-08-23 09:56:08.777 257031 INFO cinder.volume.manager [req-3b16bf94-cbd6-4d3e-b4b3-6a6da55e994a 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Created volume successfully. >2018-08-23 09:56:09.876 257256 INFO cinder.volume.flows.manager.create_volume [req-71c2a0ea-de7b-4305-a9a9-7b30df985afc 474525c8d24a4d398c71541c0e462579 e77b107cb95342b593ce0d5f03ed9600 - default default] Volume 10638846-7599-4b4f-98cc-11a0cf8c22fa: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-10638846-7599-4b4f-98cc-11a0cf8c22fa'} >2018-08-23 09:56:10.591 257256 INFO cinder.volume.flows.manager.create_volume [req-71c2a0ea-de7b-4305-a9a9-7b30df985afc 474525c8d24a4d398c71541c0e462579 e77b107cb95342b593ce0d5f03ed9600 - default default] Volume volume-10638846-7599-4b4f-98cc-11a0cf8c22fa (10638846-7599-4b4f-98cc-11a0cf8c22fa): created successfully >2018-08-23 09:56:10.594 257256 INFO cinder.volume.manager [req-71c2a0ea-de7b-4305-a9a9-7b30df985afc 474525c8d24a4d398c71541c0e462579 e77b107cb95342b593ce0d5f03ed9600 - default default] Created volume successfully. >2018-08-23 09:56:14.557 257256 INFO cinder.volume.manager [req-7e6ecf34-f290-42f9-bd39-92402c247b26 474525c8d24a4d398c71541c0e462579 e77b107cb95342b593ce0d5f03ed9600 - default default] Deleted volume successfully. >2018-08-23 09:56:14.598 257256 INFO cinder.volume.manager [req-63bd5717-bb52-4dfc-9097-41a82b3e1072 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Deleted volume successfully. >2018-08-23 09:56:14.728 257256 INFO cinder.volume.manager [req-48b3beca-e421-4f74-85ae-a0ef732e52cd 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Deleted volume successfully. >2018-08-23 09:56:14.963 257031 INFO cinder.volume.manager [req-1ccb8824-a0b2-4974-988e-9cc7ae150e8e 1541abe5e0bc441c87e33435176711fa fb215c846b724d4fbe7d598128310017 - default default] Deleted volume successfully. >2018-08-23 09:56:20.278 257256 INFO cinder.volume.flows.manager.create_volume [req-fb874d8b-8d22-4ddc-8ad7-930f078dae7e e111c568e8504cf6a14930fd3fd97edf 5c7928f0cdef4abfbf2d9dae2c0a8bf7 - default default] Volume d09dafb7-da0c-45e5-a23a-4e25d319b8df: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d09dafb7-da0c-45e5-a23a-4e25d319b8df'} >2018-08-23 09:56:20.887 257256 INFO cinder.volume.flows.manager.create_volume [req-fb874d8b-8d22-4ddc-8ad7-930f078dae7e e111c568e8504cf6a14930fd3fd97edf 5c7928f0cdef4abfbf2d9dae2c0a8bf7 - default default] Volume volume-d09dafb7-da0c-45e5-a23a-4e25d319b8df (d09dafb7-da0c-45e5-a23a-4e25d319b8df): created successfully >2018-08-23 09:56:20.891 257256 INFO cinder.volume.manager [req-fb874d8b-8d22-4ddc-8ad7-930f078dae7e e111c568e8504cf6a14930fd3fd97edf 5c7928f0cdef4abfbf2d9dae2c0a8bf7 - default default] Created volume successfully. >2018-08-23 09:56:25.403 257256 INFO cinder.volume.manager [req-d43ac1da-6457-4ee6-ac10-10aff4ba252b e111c568e8504cf6a14930fd3fd97edf 5c7928f0cdef4abfbf2d9dae2c0a8bf7 - default default] Deleted volume successfully. >2018-08-23 09:56:32.428 257256 INFO cinder.volume.flows.manager.create_volume [req-e349df25-a424-4769-9b18-90d5572ff128 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] Volume 68531b1d-51f7-47b0-bd9e-1a4dcaa06e4c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-68531b1d-51f7-47b0-bd9e-1a4dcaa06e4c'} >2018-08-23 09:56:32.952 257256 INFO cinder.volume.flows.manager.create_volume [req-e349df25-a424-4769-9b18-90d5572ff128 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] Volume volume-68531b1d-51f7-47b0-bd9e-1a4dcaa06e4c (68531b1d-51f7-47b0-bd9e-1a4dcaa06e4c): created successfully >2018-08-23 09:56:32.955 257256 INFO cinder.volume.manager [req-e349df25-a424-4769-9b18-90d5572ff128 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] Created volume successfully. >2018-08-23 09:56:35.352 257256 INFO cinder.volume.manager [req-1b8e507e-6fc2-4a64-848f-35fb3402eb18 113aee42438e49689dbc1fe846c51cde 94e28852005743b1b6b717fabf12758e - default default] Transfer volume completed successfully. >2018-08-23 09:56:37.656 257256 INFO cinder.volume.manager [req-e280e00e-fd5f-4cc0-9725-50bcd6a75de8 b91d854d4b7645f3a0e77eacab8947b5 9bfec01a9ed5421fbe68d599bc156310 - default default] Deleted volume successfully. >2018-08-23 09:56:38.603 257256 INFO cinder.volume.flows.manager.create_volume [req-c6e2e955-4b5b-459e-a94d-0be9ddc292a7 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] Volume d46a4ea7-2070-488b-82f5-8dd73a970cef: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d46a4ea7-2070-488b-82f5-8dd73a970cef'} >2018-08-23 09:56:39.124 257256 INFO cinder.volume.flows.manager.create_volume [req-c6e2e955-4b5b-459e-a94d-0be9ddc292a7 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] Volume volume-d46a4ea7-2070-488b-82f5-8dd73a970cef (d46a4ea7-2070-488b-82f5-8dd73a970cef): created successfully >2018-08-23 09:56:39.128 257256 INFO cinder.volume.manager [req-c6e2e955-4b5b-459e-a94d-0be9ddc292a7 f08a4bc0051d4598840506a0d0627f22 4e2c6413623c4fe5b5083a76bcf37cf0 - default default] Created volume successfully. >2018-08-23 09:56:41.909 257256 INFO cinder.volume.manager [req-efac5d00-5399-46f9-b9e6-7ca79ca0cf8c b91d854d4b7645f3a0e77eacab8947b5 9bfec01a9ed5421fbe68d599bc156310 - default default] Deleted volume successfully. >2018-08-23 09:56:49.042 257256 INFO cinder.volume.flows.manager.create_volume [req-7f3eea39-9eb4-44ce-b318-6ff0ec1c63a7 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Volume 380786b3-2bd4-4a75-bf70-b4a3baf85483: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-380786b3-2bd4-4a75-bf70-b4a3baf85483'} >2018-08-23 09:56:49.735 257256 INFO cinder.volume.flows.manager.create_volume [req-7f3eea39-9eb4-44ce-b318-6ff0ec1c63a7 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Volume volume-380786b3-2bd4-4a75-bf70-b4a3baf85483 (380786b3-2bd4-4a75-bf70-b4a3baf85483): created successfully >2018-08-23 09:56:49.739 257256 INFO cinder.volume.manager [req-7f3eea39-9eb4-44ce-b318-6ff0ec1c63a7 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Created volume successfully. >2018-08-23 09:56:56.424 257256 INFO cinder.volume.manager [req-82e4a458-6321-46b3-b570-6961c05e8df4 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Attaching volume 380786b3-2bd4-4a75-bf70-b4a3baf85483 to instance d0d9f582-81d9-4665-bf1d-86e17ddf7772 at mountpoint /dev/vdb on host None. >2018-08-23 09:56:56.496 257256 INFO cinder.volume.manager [req-82e4a458-6321-46b3-b570-6961c05e8df4 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Attach volume completed successfully. >2018-08-23 09:56:57.195 257256 INFO cinder.volume.manager [req-ebfe02f9-b882-46e0-8b80-48627028a98f 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Detaching volume 380786b3-2bd4-4a75-bf70-b4a3baf85483 from instance d0d9f582-81d9-4665-bf1d-86e17ddf7772. >2018-08-23 09:56:57.279 257256 INFO cinder.volume.manager [req-ebfe02f9-b882-46e0-8b80-48627028a98f 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Detach volume completed successfully. >2018-08-23 09:57:03.834 257256 INFO cinder.volume.manager [req-de6c3067-4482-4a56-b448-8165d03444e2 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Attaching volume 380786b3-2bd4-4a75-bf70-b4a3baf85483 to instance b3d0fa0a-e75e-4f8a-9d94-37fb1e240627 at mountpoint /dev/vdb on host None. >2018-08-23 09:57:03.904 257256 INFO cinder.volume.manager [req-de6c3067-4482-4a56-b448-8165d03444e2 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Attach volume completed successfully. >2018-08-23 09:57:04.890 257256 INFO cinder.volume.manager [req-df7e390c-da97-4c3f-b124-4a80a79896a9 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Detaching volume 380786b3-2bd4-4a75-bf70-b4a3baf85483 from instance b3d0fa0a-e75e-4f8a-9d94-37fb1e240627. >2018-08-23 09:57:04.984 257256 INFO cinder.volume.manager [req-df7e390c-da97-4c3f-b124-4a80a79896a9 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Detach volume completed successfully. >2018-08-23 09:57:19.989 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 09:57:38.378 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-ca734d0e-77e3-4fea-923a-0c540b1e2000 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 09:57:38.716 257256 INFO cinder.volume.manager [req-ca734d0e-77e3-4fea-923a-0c540b1e2000 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Copy volume to image completed successfully. >2018-08-23 09:57:41.515 257256 INFO cinder.volume.manager [req-1e29cdb8-1646-4cf7-8487-47dee661239d 0f2ee72dc5454b45a3ea66ca13131e89 c3b3ae44e96c488c9b0ffa5e5be1978b - default default] Deleted volume successfully. >2018-08-23 09:57:47.827 257256 INFO cinder.volume.flows.manager.create_volume [req-232479fc-277e-404c-8bd3-04acc9dcc331 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Volume 41d35f90-dd4f-47da-8792-0cc64ec3b225: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-41d35f90-dd4f-47da-8792-0cc64ec3b225'} >2018-08-23 09:57:48.377 257256 INFO cinder.volume.flows.manager.create_volume [req-232479fc-277e-404c-8bd3-04acc9dcc331 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Volume volume-41d35f90-dd4f-47da-8792-0cc64ec3b225 (41d35f90-dd4f-47da-8792-0cc64ec3b225): created successfully >2018-08-23 09:57:48.382 257256 INFO cinder.volume.manager [req-232479fc-277e-404c-8bd3-04acc9dcc331 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Created volume successfully. >2018-08-23 09:58:10.867 257256 INFO cinder.volume.manager [req-7c8dd146-082a-4498-8fa6-cdcb20965bfb dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Initialize volume connection completed successfully. >2018-08-23 09:58:15.652 257256 INFO cinder.volume.manager [req-167dbfaf-4bc5-4c9d-b111-99c958eec63d dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Attaching volume 41d35f90-dd4f-47da-8792-0cc64ec3b225 to instance e26cca4e-f069-4696-a51e-8d4511f8453b at mountpoint /dev/vdb on host None. >2018-08-23 09:58:15.728 257256 INFO cinder.volume.manager [req-167dbfaf-4bc5-4c9d-b111-99c958eec63d dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Attach volume completed successfully. >2018-08-23 09:58:17.395 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e9d8c3a5-3043-4f4d-a149-6ff1caf06c17 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Unable to get remote copy information for volume volume-41d35f90-dd4f-47da-8792-0cc64ec3b225 >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova [req-e9d8c3a5-3043-4f4d-a149-6ff1caf06c17 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Failed to notify nova on events: [{'tag': '41d35f90-dd4f-47da-8792-0cc64ec3b225', 'name': 'volume-extended', 'server_uuid': 'e26cca4e-f069-4696-a51e-8d4511f8453b'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-7b1f8e29-71aa-4479-86b5-8fa0883d8857) >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova return_raw=True) >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-7b1f8e29-71aa-4479-86b5-8fa0883d8857) >2018-08-23 09:58:18.381 257256 ERROR cinder.compute.nova >2018-08-23 09:58:18.415 257256 INFO cinder.volume.manager [req-e9d8c3a5-3043-4f4d-a149-6ff1caf06c17 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Extend volume completed successfully. >2018-08-23 10:03:30.236 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-e3be2ea0-ab93-4d6f-b185-b7242f997683 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:03:30.511 257256 INFO cinder.volume.manager [req-e3be2ea0-ab93-4d6f-b185-b7242f997683 dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Terminate volume connection completed successfully. >2018-08-23 10:03:30.909 257256 INFO cinder.volume.manager [req-e4a236b2-3335-400b-a7d9-2e7e1c988efa dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Detaching volume 41d35f90-dd4f-47da-8792-0cc64ec3b225 from instance e26cca4e-f069-4696-a51e-8d4511f8453b. >2018-08-23 10:03:30.998 257256 INFO cinder.volume.manager [req-e4a236b2-3335-400b-a7d9-2e7e1c988efa dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Detach volume completed successfully. >2018-08-23 10:03:35.524 257256 INFO cinder.volume.manager [req-ce5c3f29-dc3e-4cac-9760-02d487dc22dc dc1d37665150414f826632db1a7c4777 00539a562d484e1c99982e9db2ca9d11 - default default] Deleted volume successfully. >2018-08-23 10:03:41.240 257256 INFO cinder.volume.flows.manager.create_volume [req-e2e8acf3-45b3-474b-9de2-6e25493340ba 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] Volume b6b0a829-9c12-409a-986c-e518b41c503e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b6b0a829-9c12-409a-986c-e518b41c503e'} >2018-08-23 10:03:41.786 257256 INFO cinder.volume.flows.manager.create_volume [req-e2e8acf3-45b3-474b-9de2-6e25493340ba 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] Volume volume-b6b0a829-9c12-409a-986c-e518b41c503e (b6b0a829-9c12-409a-986c-e518b41c503e): created successfully >2018-08-23 10:03:41.790 257256 INFO cinder.volume.manager [req-e2e8acf3-45b3-474b-9de2-6e25493340ba 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] Created volume successfully. >2018-08-23 10:03:43.439 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-eb2f737e-b41a-49fd-ae72-a9b3096401bf 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] Unable to get remote copy information for volume volume-b6b0a829-9c12-409a-986c-e518b41c503e >2018-08-23 10:03:43.719 257256 INFO cinder.volume.manager [req-eb2f737e-b41a-49fd-ae72-a9b3096401bf 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] Extend volume completed successfully. >2018-08-23 10:03:45.892 257256 INFO cinder.volume.manager [req-aef6676b-286b-44d5-a96c-0547549b9983 5b28ec48fcfc4b9dba7c4c0fd5e73d42 f7b7d658ccd94a9eb10aa9768b2ff5fa - default default] Deleted volume successfully. >2018-08-23 10:03:50.675 257256 INFO cinder.volume.flows.manager.create_volume [req-9b3c7ba3-693b-4b61-b920-af66debeb0da 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume 1ed5a1d9-a72e-47c0-876d-8c4b8abf29d8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1ed5a1d9-a72e-47c0-876d-8c4b8abf29d8'} >2018-08-23 10:03:51.200 257256 INFO cinder.volume.flows.manager.create_volume [req-9b3c7ba3-693b-4b61-b920-af66debeb0da 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume volume-1ed5a1d9-a72e-47c0-876d-8c4b8abf29d8 (1ed5a1d9-a72e-47c0-876d-8c4b8abf29d8): created successfully >2018-08-23 10:03:51.204 257256 INFO cinder.volume.manager [req-9b3c7ba3-693b-4b61-b920-af66debeb0da 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Created volume successfully. >2018-08-23 10:03:53.445 257256 INFO cinder.volume.flows.manager.create_volume [req-45d71ae7-f8d3-44ae-a35f-f64d09d47059 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume 59da03c7-0ba0-4337-af7d-f30b7328e86c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-59da03c7-0ba0-4337-af7d-f30b7328e86c'} >2018-08-23 10:03:53.971 257256 INFO cinder.volume.flows.manager.create_volume [req-45d71ae7-f8d3-44ae-a35f-f64d09d47059 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume volume-59da03c7-0ba0-4337-af7d-f30b7328e86c (59da03c7-0ba0-4337-af7d-f30b7328e86c): created successfully >2018-08-23 10:03:53.975 257256 INFO cinder.volume.manager [req-45d71ae7-f8d3-44ae-a35f-f64d09d47059 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Created volume successfully. >2018-08-23 10:03:56.190 257256 INFO cinder.volume.manager [req-ff0bf3be-626d-4647-9a2b-c17f31469895 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Deleted volume successfully. >2018-08-23 10:03:58.121 257256 INFO cinder.volume.manager [req-276fc352-397b-4a5a-bb30-1c45ac626ec8 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Deleted volume successfully. >2018-08-23 10:03:59.516 257256 INFO cinder.volume.flows.manager.create_volume [req-9532a80b-146e-4a0e-ac41-2f4b8f039d9a 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume e70b3ae8-4df0-47fb-ae3b-4b5cf5e45fc1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e70b3ae8-4df0-47fb-ae3b-4b5cf5e45fc1'} >2018-08-23 10:04:00.058 257256 INFO cinder.volume.flows.manager.create_volume [req-9532a80b-146e-4a0e-ac41-2f4b8f039d9a 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume volume-e70b3ae8-4df0-47fb-ae3b-4b5cf5e45fc1 (e70b3ae8-4df0-47fb-ae3b-4b5cf5e45fc1): created successfully >2018-08-23 10:04:00.062 257256 INFO cinder.volume.manager [req-9532a80b-146e-4a0e-ac41-2f4b8f039d9a 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Created volume successfully. >2018-08-23 10:04:01.140 257256 INFO cinder.volume.flows.manager.create_volume [req-aa61a00c-a9d0-4144-8e0e-c2f57f7afe00 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume a05aab85-0160-48ff-8e97-106b6cb6b5e5: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a05aab85-0160-48ff-8e97-106b6cb6b5e5', 'source_volid': 'e70b3ae8-4df0-47fb-ae3b-4b5cf5e45fc1', 'source_volstatus': u'available'} >2018-08-23 10:04:07.958 257256 INFO cinder.volume.flows.manager.create_volume [req-aa61a00c-a9d0-4144-8e0e-c2f57f7afe00 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume volume-a05aab85-0160-48ff-8e97-106b6cb6b5e5 (a05aab85-0160-48ff-8e97-106b6cb6b5e5): created successfully >2018-08-23 10:04:07.962 257256 INFO cinder.volume.manager [req-aa61a00c-a9d0-4144-8e0e-c2f57f7afe00 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Created volume successfully. >2018-08-23 10:04:09.391 257256 INFO cinder.volume.flows.manager.create_volume [req-87ef4347-dc24-4547-8350-ef327f2c5a50 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume ea901e89-2abc-4421-bbd0-cc5e45ef8ba7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ea901e89-2abc-4421-bbd0-cc5e45ef8ba7'} >2018-08-23 10:04:09.919 257256 INFO cinder.volume.flows.manager.create_volume [req-87ef4347-dc24-4547-8350-ef327f2c5a50 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume volume-ea901e89-2abc-4421-bbd0-cc5e45ef8ba7 (ea901e89-2abc-4421-bbd0-cc5e45ef8ba7): created successfully >2018-08-23 10:04:09.925 257256 INFO cinder.volume.manager [req-87ef4347-dc24-4547-8350-ef327f2c5a50 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Created volume successfully. >2018-08-23 10:04:11.828 257256 INFO cinder.volume.manager [req-3c159c28-55c4-4950-a3cd-ad6288f83e8b 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Deleted volume successfully. >2018-08-23 10:04:16.123 257256 INFO cinder.volume.manager [req-48382be2-bf78-47ba-b532-c36f53176b2f 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Deleted volume successfully. >2018-08-23 10:04:17.528 257256 INFO cinder.volume.flows.manager.create_volume [req-9afb89f8-378b-4430-b62e-ae2633ec83ea 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume b7185666-72cf-4dd6-8148-1f70beb3a891: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-b7185666-72cf-4dd6-8148-1f70beb3a891', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54c79c10>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 10:04:18.553 257256 INFO cinder.image.image_utils [req-9afb89f8-378b-4430-b62e-ae2633ec83ea 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Image download 12.00 MB at 11.72 MB/s >2018-08-23 10:04:27.348 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 10:04:32.279 257256 INFO cinder.image.image_utils [req-9afb89f8-378b-4430-b62e-ae2633ec83ea 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 10:04:34.129 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-9afb89f8-378b-4430-b62e-ae2633ec83ea 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:04:34.767 257256 INFO cinder.volume.flows.manager.create_volume [req-9afb89f8-378b-4430-b62e-ae2633ec83ea 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume volume-b7185666-72cf-4dd6-8148-1f70beb3a891 (b7185666-72cf-4dd6-8148-1f70beb3a891): created successfully >2018-08-23 10:04:34.771 257256 INFO cinder.volume.manager [req-9afb89f8-378b-4430-b62e-ae2633ec83ea 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Created volume successfully. >2018-08-23 10:04:36.040 257256 INFO cinder.volume.flows.manager.create_volume [req-2a80ff86-df3d-43df-9d74-26b9613d2285 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume 495690f9-e300-42bb-9290-4d943e9e8ac5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-495690f9-e300-42bb-9290-4d943e9e8ac5'} >2018-08-23 10:04:36.576 257256 INFO cinder.volume.flows.manager.create_volume [req-2a80ff86-df3d-43df-9d74-26b9613d2285 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Volume volume-495690f9-e300-42bb-9290-4d943e9e8ac5 (495690f9-e300-42bb-9290-4d943e9e8ac5): created successfully >2018-08-23 10:04:36.580 257256 INFO cinder.volume.manager [req-2a80ff86-df3d-43df-9d74-26b9613d2285 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Created volume successfully. >2018-08-23 10:04:38.334 257256 INFO cinder.volume.manager [req-60b6656f-95a0-4b0f-89d3-b5859d87713b 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Deleted volume successfully. >2018-08-23 10:04:39.556 257256 INFO cinder.volume.manager [req-fd019472-ed20-407a-a288-35455b5222c4 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Deleted volume successfully. >2018-08-23 10:04:40.826 257256 INFO cinder.volume.manager [req-10e49b4f-55a5-4461-ae7a-83464b028b11 5457388f4904425fae7ab8f0486b0472 d405d80a5d674e0fb688aaccaf4e0607 - default default] Deleted volume successfully. >2018-08-23 10:04:50.046 257256 INFO cinder.volume.flows.manager.create_volume [req-75c7d291-7ac2-4b4c-8313-ab11c23c2786 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Volume ba14d115-c0dc-4641-9ee8-77f8316bbd0c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ba14d115-c0dc-4641-9ee8-77f8316bbd0c'} >2018-08-23 10:04:50.571 257256 INFO cinder.volume.flows.manager.create_volume [req-75c7d291-7ac2-4b4c-8313-ab11c23c2786 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Volume volume-ba14d115-c0dc-4641-9ee8-77f8316bbd0c (ba14d115-c0dc-4641-9ee8-77f8316bbd0c): created successfully >2018-08-23 10:04:50.575 257256 INFO cinder.volume.manager [req-75c7d291-7ac2-4b4c-8313-ab11c23c2786 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Created volume successfully. >2018-08-23 10:04:52.083 257256 INFO cinder.volume.flows.manager.create_volume [req-b72f6bae-fca7-48e2-bf18-efccc05fe764 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Volume 068f1fc7-f1ef-4226-b67b-3502c12ad610: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-068f1fc7-f1ef-4226-b67b-3502c12ad610'} >2018-08-23 10:04:52.907 257256 INFO cinder.volume.flows.manager.create_volume [req-b72f6bae-fca7-48e2-bf18-efccc05fe764 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Volume volume-068f1fc7-f1ef-4226-b67b-3502c12ad610 (068f1fc7-f1ef-4226-b67b-3502c12ad610): created successfully >2018-08-23 10:04:52.911 257256 INFO cinder.volume.manager [req-b72f6bae-fca7-48e2-bf18-efccc05fe764 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Created volume successfully. >2018-08-23 10:04:53.926 257256 INFO cinder.volume.flows.manager.create_volume [req-a6a36dae-9b14-4e64-854b-1bf73e19002a bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Volume 192afe0b-c9f3-4a84-a8a5-e7dc0315b57b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-192afe0b-c9f3-4a84-a8a5-e7dc0315b57b'} >2018-08-23 10:04:55.777 257256 INFO cinder.volume.flows.manager.create_volume [req-a6a36dae-9b14-4e64-854b-1bf73e19002a bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Volume volume-192afe0b-c9f3-4a84-a8a5-e7dc0315b57b (192afe0b-c9f3-4a84-a8a5-e7dc0315b57b): created successfully >2018-08-23 10:04:55.781 257256 INFO cinder.volume.manager [req-a6a36dae-9b14-4e64-854b-1bf73e19002a bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Created volume successfully. >2018-08-23 10:05:02.756 257256 INFO cinder.volume.manager [req-91ef739a-6482-48fe-a62c-70e48c168677 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Deleted volume successfully. >2018-08-23 10:05:03.047 257256 INFO cinder.volume.manager [req-2c853dab-54e5-41b2-91be-95f4b6b6780f bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Deleted volume successfully. >2018-08-23 10:05:03.171 257256 INFO cinder.volume.manager [req-2a32c7e8-9913-487a-8545-6e0fa3a6abf5 bf97c8836b9746a8bed600469061d367 c03ebc8504f34f128ef839293b63ca29 - default default] Deleted volume successfully. >2018-08-23 10:05:07.531 257256 INFO cinder.volume.flows.manager.create_volume [req-b7d5c8ba-6fd6-44fc-8639-35a07b208057 20b0428cd13544398021dc3bb12e0d2f d5c76be52e2b4fe08286d2cc03b07dc5 - default default] Volume 17c36b25-f405-4a6d-8377-e8248999aa3e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-17c36b25-f405-4a6d-8377-e8248999aa3e'} >2018-08-23 10:05:08.387 257256 INFO cinder.volume.flows.manager.create_volume [req-b7d5c8ba-6fd6-44fc-8639-35a07b208057 20b0428cd13544398021dc3bb12e0d2f d5c76be52e2b4fe08286d2cc03b07dc5 - default default] Volume volume-17c36b25-f405-4a6d-8377-e8248999aa3e (17c36b25-f405-4a6d-8377-e8248999aa3e): created successfully >2018-08-23 10:05:08.391 257256 INFO cinder.volume.manager [req-b7d5c8ba-6fd6-44fc-8639-35a07b208057 20b0428cd13544398021dc3bb12e0d2f d5c76be52e2b4fe08286d2cc03b07dc5 - default default] Created volume successfully. >2018-08-23 10:05:28.837 257256 INFO cinder.volume.manager [req-38b278de-fd23-4f9a-8612-4d49c776c37d 20b0428cd13544398021dc3bb12e0d2f d5c76be52e2b4fe08286d2cc03b07dc5 - default default] Deleted volume successfully. >2018-08-23 10:05:33.654 257256 INFO cinder.volume.flows.manager.create_volume [req-44916211-e906-4124-ab6d-a23544a89dfb 0c496b0845b84a01abb41f3162dab88c b89c2689fe1a4807b408f185174e8210 - default default] Volume e05ecf09-feb2-4b1e-a48f-7bec23073f08: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e05ecf09-feb2-4b1e-a48f-7bec23073f08'} >2018-08-23 10:05:34.532 257256 INFO cinder.volume.flows.manager.create_volume [req-44916211-e906-4124-ab6d-a23544a89dfb 0c496b0845b84a01abb41f3162dab88c b89c2689fe1a4807b408f185174e8210 - default default] Volume volume-e05ecf09-feb2-4b1e-a48f-7bec23073f08 (e05ecf09-feb2-4b1e-a48f-7bec23073f08): created successfully >2018-08-23 10:05:34.536 257256 INFO cinder.volume.manager [req-44916211-e906-4124-ab6d-a23544a89dfb 0c496b0845b84a01abb41f3162dab88c b89c2689fe1a4807b408f185174e8210 - default default] Created volume successfully. >2018-08-23 10:05:38.050 257256 INFO cinder.volume.manager [req-0b6a0750-c44a-4df3-8969-f37ba6b25672 0c496b0845b84a01abb41f3162dab88c b89c2689fe1a4807b408f185174e8210 - default default] Deleted volume successfully. >2018-08-23 10:06:26.312 257256 INFO cinder.volume.flows.manager.create_volume [req-bf117181-549e-404b-b4a0-573de024abc9 c22e0738058d4a2aae4a8e7470e507b8 f28024db34a841c29dc5e1acf3ae0204 - default default] Volume ee30de3c-d612-42ba-977d-261fb953e711: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ee30de3c-d612-42ba-977d-261fb953e711'} >2018-08-23 10:06:26.868 257256 INFO cinder.volume.flows.manager.create_volume [req-bf117181-549e-404b-b4a0-573de024abc9 c22e0738058d4a2aae4a8e7470e507b8 f28024db34a841c29dc5e1acf3ae0204 - default default] Volume volume-ee30de3c-d612-42ba-977d-261fb953e711 (ee30de3c-d612-42ba-977d-261fb953e711): created successfully >2018-08-23 10:06:26.871 257256 INFO cinder.volume.manager [req-bf117181-549e-404b-b4a0-573de024abc9 c22e0738058d4a2aae4a8e7470e507b8 f28024db34a841c29dc5e1acf3ae0204 - default default] Created volume successfully. >2018-08-23 10:06:29.928 257256 INFO cinder.volume.manager [req-4fea0396-49c5-4cc6-b1ea-7c273e9c393b c22e0738058d4a2aae4a8e7470e507b8 f28024db34a841c29dc5e1acf3ae0204 - default default] Deleted volume successfully. >2018-08-23 10:06:35.244 257256 INFO cinder.volume.flows.manager.create_volume [req-d85bd930-ff0e-4a3d-81a7-b4494207fdac 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Volume 3e832119-dfbe-4944-968b-c20878e44774: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-3e832119-dfbe-4944-968b-c20878e44774', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab542ebf90>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 10:06:36.392 257256 INFO cinder.image.image_utils [req-d85bd930-ff0e-4a3d-81a7-b4494207fdac 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Image download 12.00 MB at 10.47 MB/s >2018-08-23 10:06:45.342 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 10:06:50.172 257256 INFO cinder.image.image_utils [req-d85bd930-ff0e-4a3d-81a7-b4494207fdac 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-23 10:06:52.033 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-d85bd930-ff0e-4a3d-81a7-b4494207fdac 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:06:52.381 257256 INFO cinder.volume.flows.manager.create_volume [req-d85bd930-ff0e-4a3d-81a7-b4494207fdac 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Volume volume-3e832119-dfbe-4944-968b-c20878e44774 (3e832119-dfbe-4944-968b-c20878e44774): created successfully >2018-08-23 10:06:52.385 257256 INFO cinder.volume.manager [req-d85bd930-ff0e-4a3d-81a7-b4494207fdac 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Created volume successfully. >2018-08-23 10:06:53.821 257256 INFO cinder.volume.flows.manager.create_volume [req-6c4d3e7b-dd36-48ee-a149-169caaa4ff47 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Volume e610cb2a-ae89-4285-b124-4732c1b09830: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e610cb2a-ae89-4285-b124-4732c1b09830', 'source_volid': '3e832119-dfbe-4944-968b-c20878e44774', 'source_volstatus': u'available'} >2018-08-23 10:07:00.532 257256 INFO cinder.volume.flows.manager.create_volume [req-6c4d3e7b-dd36-48ee-a149-169caaa4ff47 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Volume volume-e610cb2a-ae89-4285-b124-4732c1b09830 (e610cb2a-ae89-4285-b124-4732c1b09830): created successfully >2018-08-23 10:07:00.536 257256 INFO cinder.volume.manager [req-6c4d3e7b-dd36-48ee-a149-169caaa4ff47 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Created volume successfully. >2018-08-23 10:07:04.185 257256 INFO cinder.volume.manager [req-04ab2eec-c3e7-429d-95c1-d067c497d165 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Deleted volume successfully. >2018-08-23 10:07:06.139 257256 INFO cinder.volume.manager [req-bd4fda93-f3c4-4d69-8a92-da820ad2bb5b 5c5ae209144c424984e1d3d89c06313a 508a4b6a23ab4735be7aba16ef28746f - default default] Deleted volume successfully. >2018-08-23 10:07:10.996 257256 INFO cinder.volume.flows.manager.create_volume [req-7e3e8f39-1af2-4845-a96f-086f85168e21 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Volume e7846c70-5b45-4f78-94c0-c6f35e3fe6be: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e7846c70-5b45-4f78-94c0-c6f35e3fe6be'} >2018-08-23 10:07:11.521 257256 INFO cinder.volume.flows.manager.create_volume [req-7e3e8f39-1af2-4845-a96f-086f85168e21 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Volume volume-e7846c70-5b45-4f78-94c0-c6f35e3fe6be (e7846c70-5b45-4f78-94c0-c6f35e3fe6be): created successfully >2018-08-23 10:07:11.525 257256 INFO cinder.volume.manager [req-7e3e8f39-1af2-4845-a96f-086f85168e21 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Created volume successfully. >2018-08-23 10:07:13.284 257031 INFO cinder.volume.flows.manager.create_volume [req-f2cc4850-7913-4aef-a7e3-d911063e32b8 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Volume d2c7910c-9231-4b50-95a9-efb88d8a2a2b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d2c7910c-9231-4b50-95a9-efb88d8a2a2b'} >2018-08-23 10:07:13.813 257031 INFO cinder.volume.flows.manager.create_volume [req-f2cc4850-7913-4aef-a7e3-d911063e32b8 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Volume volume-d2c7910c-9231-4b50-95a9-efb88d8a2a2b (d2c7910c-9231-4b50-95a9-efb88d8a2a2b): created successfully >2018-08-23 10:07:13.817 257031 INFO cinder.volume.manager [req-f2cc4850-7913-4aef-a7e3-d911063e32b8 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Created volume successfully. >2018-08-23 10:07:15.371 257256 INFO cinder.volume.flows.manager.create_volume [req-66db815e-232f-4f49-b2d7-ada1007ca4cf 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Volume 45a04102-cef9-4321-8cae-e0385ce49d08: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-45a04102-cef9-4321-8cae-e0385ce49d08'} >2018-08-23 10:07:15.905 257256 INFO cinder.volume.flows.manager.create_volume [req-66db815e-232f-4f49-b2d7-ada1007ca4cf 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Volume volume-45a04102-cef9-4321-8cae-e0385ce49d08 (45a04102-cef9-4321-8cae-e0385ce49d08): created successfully >2018-08-23 10:07:15.910 257256 INFO cinder.volume.manager [req-66db815e-232f-4f49-b2d7-ada1007ca4cf 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Created volume successfully. >2018-08-23 10:07:19.175 257256 INFO cinder.volume.manager [req-04ad94b8-7f8e-4988-8518-740fa85d1431 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Deleted volume successfully. >2018-08-23 10:07:19.196 257031 INFO cinder.volume.manager [req-cc836eb3-8f7e-4d9a-8294-c573a9ae8ab6 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Deleted volume successfully. >2018-08-23 10:07:19.415 257256 INFO cinder.volume.manager [req-0ccb8643-2b4d-4ee9-bdb2-8b126f92893f 5b27360a44c1497f9a68720ceb8da827 bdd9458e856b43d29f21bbd9ba516131 - default default] Deleted volume successfully. >2018-08-23 10:07:39.500 257031 INFO cinder.volume.flows.manager.create_volume [req-8353ffd3-735e-480a-b73e-c60b4ae41741 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Volume ea364890-8c5f-409f-830e-cb6b5a246ccd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ea364890-8c5f-409f-830e-cb6b5a246ccd'} >2018-08-23 10:07:40.049 257031 INFO cinder.volume.flows.manager.create_volume [req-8353ffd3-735e-480a-b73e-c60b4ae41741 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Volume volume-ea364890-8c5f-409f-830e-cb6b5a246ccd (ea364890-8c5f-409f-830e-cb6b5a246ccd): created successfully >2018-08-23 10:07:40.053 257031 INFO cinder.volume.manager [req-8353ffd3-735e-480a-b73e-c60b4ae41741 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Created volume successfully. >2018-08-23 10:07:42.150 257256 INFO cinder.volume.flows.manager.create_volume [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Volume ff3eabc5-4a20-4ca3-97fb-363259b7ea28: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28'} >2018-08-23 10:07:42.706 257256 INFO cinder.volume.flows.manager.create_volume [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Volume volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28 (ff3eabc5-4a20-4ca3-97fb-363259b7ea28): created successfully >2018-08-23 10:07:42.710 257256 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Created volume successfully. >2018-08-23 10:07:50.173 257256 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Initialize volume connection completed successfully. >2018-08-23 10:07:50.567 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 10:07:54.040 257031 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Initialize volume connection completed successfully. >2018-08-23 10:07:54.422 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 10:08:00.370 257031 INFO cinder.volume.utils [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Volume copy 1024.00 MB at 342.34 MB/s >2018-08-23 10:08:02.254 257256 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Terminate volume connection completed successfully. >2018-08-23 10:08:02.298 257256 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Remove volume export completed successfully. >2018-08-23 10:08:04.138 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:08:04.419 257031 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Terminate volume connection completed successfully. >2018-08-23 10:08:04.454 257031 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Remove volume export completed successfully. >2018-08-23 10:08:04.855 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 10:08:08.097 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Error running SSH command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 10:08:08.099 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 10:08:08.099 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Error running SSH command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 10:08:08.100 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-ea364890-8c5f-409f-830e-cb6b5a246ccd', '"volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-ea364890-8c5f-409f-830e-cb6b5a246ccd "volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-23 10:08:08.101 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Unable to rename the logical volume for volume: ea364890-8c5f-409f-830e-cb6b5a246ccd: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-ea364890-8c5f-409f-830e-cb6b5a246ccd', '"volume-ff3eabc5-4a20-4ca3-97fb-363259b7ea28"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-23 10:08:08.353 257031 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Complete-Migrate volume completed successfully. >2018-08-23 10:08:08.354 257031 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Migrate volume completed successfully. >2018-08-23 10:08:08.832 257031 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Retype volume completed successfully. >2018-08-23 10:08:09.489 257031 INFO cinder.volume.manager [req-f0e986e8-6b10-4c5d-9fe6-0fdcb395c27d 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Deleted volume successfully. >2018-08-23 10:08:10.764 257256 WARNING cinder.quota [req-332bf0e8-5539-42fc-8cd4-53f109547eb8 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1463910011 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1463910011, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 10:08:10.765 257256 WARNING cinder.quota [req-332bf0e8-5539-42fc-8cd4-53f109547eb8 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1463910011 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1463910011, it is now deprecated. Please use the default quota class for default quota. >2018-08-23 10:08:11.020 257256 INFO cinder.volume.manager [req-332bf0e8-5539-42fc-8cd4-53f109547eb8 716a7090849c48e1904acab946c953d7 46e5759049ba4097bcc8f2a703acf9ba - default default] Deleted volume successfully. >2018-08-23 10:08:31.567 257256 INFO cinder.volume.flows.manager.create_volume [req-c3f80162-78ec-452e-a086-5beb6246092a 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Volume 1dd209ae-4437-4095-afc3-36cf43bba7d6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1dd209ae-4437-4095-afc3-36cf43bba7d6'} >2018-08-23 10:08:32.095 257256 INFO cinder.volume.flows.manager.create_volume [req-c3f80162-78ec-452e-a086-5beb6246092a 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Volume volume-1dd209ae-4437-4095-afc3-36cf43bba7d6 (1dd209ae-4437-4095-afc3-36cf43bba7d6): created successfully >2018-08-23 10:08:32.100 257256 INFO cinder.volume.manager [req-c3f80162-78ec-452e-a086-5beb6246092a 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Created volume successfully. >2018-08-23 10:08:34.721 257256 INFO cinder.volume.manager [req-0050d24e-2e0b-4ef9-9b5f-83ef2fc7ee25 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Create snapshot completed successfully >2018-08-23 10:08:38.819 257256 INFO cinder.volume.manager [req-24ae7e06-a6b5-4c7c-a6b4-b2ab37713a55 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Create snapshot completed successfully >2018-08-23 10:08:41.865 257256 INFO cinder.volume.manager [req-53c10ade-c9b3-43ea-bb7b-f5940374c502 1fec518015af4c88b1f464bc8d96dc21 c4cd9534654f48279640199005d27509 - default default] Delete snapshot completed successfully. >2018-08-23 10:08:44.333 257256 INFO cinder.volume.manager [req-8695a81a-a000-4274-b79f-0177bdc5821b 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Create snapshot completed successfully >2018-08-23 10:08:47.447 257256 INFO cinder.volume.manager [req-c9a9c839-66dc-459d-8dd8-4e4c850b6ca1 1fec518015af4c88b1f464bc8d96dc21 c4cd9534654f48279640199005d27509 - default default] Delete snapshot completed successfully. >2018-08-23 10:08:54.884 257256 INFO cinder.volume.manager [req-800f57f5-2d7e-44a0-b241-7eb93013a174 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Create snapshot completed successfully >2018-08-23 10:08:58.186 257256 INFO cinder.volume.manager [req-74f71dfc-04fc-4168-b7c2-bd87c8c93dbb 1fec518015af4c88b1f464bc8d96dc21 c4cd9534654f48279640199005d27509 - default default] Delete snapshot completed successfully. >2018-08-23 10:09:06.664 257256 INFO cinder.volume.manager [req-c9b2d452-3143-41b2-bfe2-8f9fe31dc448 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Create snapshot completed successfully >2018-08-23 10:09:10.473 257256 INFO cinder.volume.manager [req-2fb387c4-b2ea-4239-918f-d0a2941054c4 1fec518015af4c88b1f464bc8d96dc21 c4cd9534654f48279640199005d27509 - default default] Delete snapshot completed successfully. >2018-08-23 10:09:14.082 257256 INFO cinder.volume.manager [req-2f5166b5-9681-4f28-aaf6-673fb7b1324f 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Delete snapshot completed successfully. >2018-08-23 10:09:16.682 257256 INFO cinder.volume.manager [req-1dcddccf-05b8-43f7-adc7-b2c5e65cc643 990e7b0db12948faae40052269cdb66c ba9471c290444d76bdc5b55980a404c1 - default default] Deleted volume successfully. >2018-08-23 10:09:22.522 257256 INFO cinder.volume.flows.manager.create_volume [req-fa91f305-ee8a-4758-a843-1fd9cb74bb3e 0d02edc17fad4ad2b8598fdb35fba5f1 2f199479c7524b01a6d08ade832293a1 - default default] Volume ea9158e7-0c40-4de0-9f9b-bd0348e64185: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ea9158e7-0c40-4de0-9f9b-bd0348e64185'} >2018-08-23 10:09:23.436 257256 INFO cinder.volume.flows.manager.create_volume [req-fa91f305-ee8a-4758-a843-1fd9cb74bb3e 0d02edc17fad4ad2b8598fdb35fba5f1 2f199479c7524b01a6d08ade832293a1 - default default] Volume volume-ea9158e7-0c40-4de0-9f9b-bd0348e64185 (ea9158e7-0c40-4de0-9f9b-bd0348e64185): created successfully >2018-08-23 10:09:23.440 257256 INFO cinder.volume.manager [req-fa91f305-ee8a-4758-a843-1fd9cb74bb3e 0d02edc17fad4ad2b8598fdb35fba5f1 2f199479c7524b01a6d08ade832293a1 - default default] Created volume successfully. >2018-08-23 10:09:27.055 257256 INFO cinder.volume.manager [req-88e88180-6dfc-41b3-b26a-12baa64fc3dc 0d02edc17fad4ad2b8598fdb35fba5f1 2f199479c7524b01a6d08ade832293a1 - default default] Create snapshot completed successfully >2018-08-23 10:09:33.322 257256 INFO cinder.volume.manager [req-ec53f942-a6c5-4fc9-8e87-3db8b7e390f9 0d02edc17fad4ad2b8598fdb35fba5f1 2f199479c7524b01a6d08ade832293a1 - default default] Delete snapshot completed successfully. >2018-08-23 10:09:36.175 257256 INFO cinder.volume.manager [req-2f277d47-d23c-4984-ac59-d976d186d887 0d02edc17fad4ad2b8598fdb35fba5f1 2f199479c7524b01a6d08ade832293a1 - default default] Deleted volume successfully. >2018-08-23 10:09:40.769 257256 INFO cinder.volume.flows.manager.create_volume [req-d5ac4851-1efc-4bf5-bed5-6497025c9290 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Volume 8eb3fa56-8b96-4d44-95f9-6edaafe84162: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8eb3fa56-8b96-4d44-95f9-6edaafe84162'} >2018-08-23 10:09:41.669 257256 INFO cinder.volume.flows.manager.create_volume [req-d5ac4851-1efc-4bf5-bed5-6497025c9290 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Volume volume-8eb3fa56-8b96-4d44-95f9-6edaafe84162 (8eb3fa56-8b96-4d44-95f9-6edaafe84162): created successfully >2018-08-23 10:09:41.673 257256 INFO cinder.volume.manager [req-d5ac4851-1efc-4bf5-bed5-6497025c9290 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Created volume successfully. >2018-08-23 10:09:45.319 257256 INFO cinder.volume.manager [req-f5b339d8-c7fa-43ed-8b3e-297d5f60a1b0 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:10:03.397 257256 INFO cinder.volume.manager [req-0391efb2-36f1-4e9f-a2dc-ce0a09bdbc0a 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Initialize volume connection completed successfully. >2018-08-23 10:10:25.795 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-0391efb2-36f1-4e9f-a2dc-ce0a09bdbc0a 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:10:26.070 257256 INFO cinder.volume.manager [req-0391efb2-36f1-4e9f-a2dc-ce0a09bdbc0a 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Terminate volume connection completed successfully. >2018-08-23 10:10:26.112 257256 INFO cinder.volume.manager [req-0391efb2-36f1-4e9f-a2dc-ce0a09bdbc0a 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Remove volume export completed successfully. >2018-08-23 10:10:30.305 257256 INFO cinder.volume.manager [req-0391efb2-36f1-4e9f-a2dc-ce0a09bdbc0a 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Deleted volume successfully. >2018-08-23 10:10:42.491 257256 INFO cinder.volume.manager [req-b527df88-ac26-4a80-888f-03d09f7363f9 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Initialize volume connection completed successfully. >2018-08-23 10:10:47.217 257256 INFO cinder.volume.manager [req-66640ddb-5102-4bf3-8181-af52f6138c1c 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Attaching volume 8eb3fa56-8b96-4d44-95f9-6edaafe84162 to instance 51fe203c-73bb-4304-a5b0-68bb5fef8c34 at mountpoint /dev/vdb on host None. >2018-08-23 10:10:47.291 257256 INFO cinder.volume.manager [req-66640ddb-5102-4bf3-8181-af52f6138c1c 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Attach volume completed successfully. >2018-08-23 10:10:54.754 257256 INFO cinder.volume.manager [req-9afc5541-f9c6-4856-9376-d978c61b392f 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:11:02.291 257256 INFO cinder.volume.manager [req-b1d71207-709c-4a48-a50e-83570119b7d0 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:11:11.046 257256 INFO cinder.volume.manager [req-f351d396-d12a-47d3-a9ce-f03f1a5cebb6 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:11:13.550 257256 INFO cinder.volume.manager [req-cb1b50eb-a376-4d6b-adba-7e5da9908371 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:11:15.676 257256 INFO cinder.volume.manager [req-941771b6-5872-4614-a742-ee650bdd7301 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:11:17.991 257256 INFO cinder.volume.manager [req-bd2acf49-a46c-42f6-b0b9-c5814fcceeb1 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:11:28.904 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-415b84f4-268b-40ad-bf80-981376f2ee8f 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:11:29.179 257256 INFO cinder.volume.manager [req-415b84f4-268b-40ad-bf80-981376f2ee8f 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Terminate volume connection completed successfully. >2018-08-23 10:11:29.330 257256 INFO cinder.volume.manager [req-d4270738-b183-4f26-b0f5-f63dc5237d27 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Detaching volume 8eb3fa56-8b96-4d44-95f9-6edaafe84162 from instance 51fe203c-73bb-4304-a5b0-68bb5fef8c34. >2018-08-23 10:11:29.414 257256 INFO cinder.volume.manager [req-d4270738-b183-4f26-b0f5-f63dc5237d27 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Detach volume completed successfully. >2018-08-23 10:11:38.634 257256 INFO cinder.volume.manager [req-77a22f1e-b085-4028-9576-b7043480c4b8 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:11:41.563 257256 INFO cinder.volume.manager [req-84c4e05e-8a15-4edb-8d0a-6aa577f3de66 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:11:49.146 257256 INFO cinder.volume.manager [req-46236544-e302-4c04-ad49-2058cd36d4a7 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:12:05.136 257256 INFO cinder.volume.manager [req-d97733e5-ca8f-481a-a6b4-1a5892c7c7cf 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Initialize volume connection completed successfully. >2018-08-23 10:12:09.759 257256 INFO cinder.volume.manager [req-98e0d6a0-0fc5-45c4-9639-3f4d4c0ab053 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Attaching volume 8eb3fa56-8b96-4d44-95f9-6edaafe84162 to instance 340046c9-ff37-4324-b31a-25a8666ea9aa at mountpoint /dev/vdb on host None. >2018-08-23 10:12:09.950 257256 INFO cinder.volume.manager [req-98e0d6a0-0fc5-45c4-9639-3f4d4c0ab053 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Attach volume completed successfully. >2018-08-23 10:12:17.183 257256 INFO cinder.volume.manager [req-2f15431f-5dfc-41cf-89bd-b97f0ca626a7 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:12:24.698 257256 INFO cinder.volume.manager [req-f0fdb937-dce6-4e4b-95eb-9b41947d3ce6 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:12:27.815 257256 INFO cinder.volume.manager [req-5bfa477d-6cb4-4db6-b429-429bd295fa19 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:12:30.550 257256 INFO cinder.volume.manager [req-363a0693-6f97-4cd4-8999-70627b4a2148 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:12:32.671 257256 INFO cinder.volume.manager [req-8462bf86-726f-4bdb-9e86-8414f5aa0286 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:12:43.602 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-33e31b5a-fdb8-4de3-9f7d-16539e46e9f8 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:12:44.092 257256 INFO cinder.volume.manager [req-33e31b5a-fdb8-4de3-9f7d-16539e46e9f8 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Terminate volume connection completed successfully. >2018-08-23 10:12:44.226 257256 INFO cinder.volume.manager [req-952c2649-cf3a-443e-908f-787a81548815 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Detaching volume 8eb3fa56-8b96-4d44-95f9-6edaafe84162 from instance 340046c9-ff37-4324-b31a-25a8666ea9aa. >2018-08-23 10:12:44.312 257256 INFO cinder.volume.manager [req-952c2649-cf3a-443e-908f-787a81548815 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Detach volume completed successfully. >2018-08-23 10:12:46.571 257031 INFO cinder.volume.flows.manager.create_volume [req-92caef28-73ba-4edf-9359-bef6ec484a8d 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Volume 1564531c-d61a-4eee-a5ba-e25431497081: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1564531c-d61a-4eee-a5ba-e25431497081'} >2018-08-23 10:12:47.104 257031 INFO cinder.volume.flows.manager.create_volume [req-92caef28-73ba-4edf-9359-bef6ec484a8d 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Volume volume-1564531c-d61a-4eee-a5ba-e25431497081 (1564531c-d61a-4eee-a5ba-e25431497081): created successfully >2018-08-23 10:12:47.108 257031 INFO cinder.volume.manager [req-92caef28-73ba-4edf-9359-bef6ec484a8d 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Created volume successfully. >2018-08-23 10:12:49.159 257031 INFO cinder.volume.manager [req-9f14bf3f-9a10-433e-9cd2-6a03d009b71d 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Create snapshot completed successfully >2018-08-23 10:12:50.083 257031 INFO cinder.volume.flows.manager.create_volume [req-d3b76a0c-12c1-484a-863b-2d8c617a9f81 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Volume b7444c9c-8404-47cc-afc4-c0b9004d5489: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': '23bddb80-2f2a-49f5-b487-82163d50be9e', 'volume_name': 'volume-b7444c9c-8404-47cc-afc4-c0b9004d5489'} >2018-08-23 10:13:16.083 257031 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 0.50 sec >2018-08-23 10:13:34.426 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-d3b76a0c-12c1-484a-863b-2d8c617a9f81 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Unable to get remote copy information for volume volume-b7444c9c-8404-47cc-afc4-c0b9004d5489 >2018-08-23 10:13:34.894 257031 INFO cinder.volume.flows.manager.create_volume [req-d3b76a0c-12c1-484a-863b-2d8c617a9f81 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Volume volume-b7444c9c-8404-47cc-afc4-c0b9004d5489 (b7444c9c-8404-47cc-afc4-c0b9004d5489): created successfully >2018-08-23 10:13:34.898 257031 INFO cinder.volume.manager [req-d3b76a0c-12c1-484a-863b-2d8c617a9f81 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Created volume successfully. >2018-08-23 10:13:38.190 257031 INFO cinder.volume.manager [req-33b9e207-eea9-4a2d-bbef-a133f6908083 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Deleted volume successfully. >2018-08-23 10:13:43.296 257256 INFO cinder.volume.manager [req-8e52e451-9a47-4cdd-95d1-9ca534a5997a 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:13:43.551 257031 INFO cinder.volume.manager [req-22f1c12d-22b0-458e-829b-cabe0f420b95 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Delete snapshot completed successfully. >2018-08-23 10:13:46.454 257031 WARNING cinder.db.sqlalchemy.api [req-8bdefe73-d497-4c86-acd9-83ef7efa1b85 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-23 10:13:46.518 257256 INFO cinder.volume.manager [req-2805ee08-84cf-404f-a940-b3f27ece63e9 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Deleted volume successfully. >2018-08-23 10:13:46.898 257031 INFO cinder.volume.manager [req-8bdefe73-d497-4c86-acd9-83ef7efa1b85 86136570f2224b038dfaf30b26cf8a19 8b174b852068464ab9f117a9aff6c46b - default default] Deleted volume successfully. >2018-08-23 10:13:52.064 257256 INFO cinder.volume.flows.manager.create_volume [req-f352c3dd-8c2e-408c-bcb2-b69e784f5a77 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Volume 22454c82-456b-4af6-b7ad-f64f8dcf5d4e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-22454c82-456b-4af6-b7ad-f64f8dcf5d4e'} >2018-08-23 10:13:52.964 257256 INFO cinder.volume.flows.manager.create_volume [req-f352c3dd-8c2e-408c-bcb2-b69e784f5a77 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Volume volume-22454c82-456b-4af6-b7ad-f64f8dcf5d4e (22454c82-456b-4af6-b7ad-f64f8dcf5d4e): created successfully >2018-08-23 10:13:52.967 257256 INFO cinder.volume.manager [req-f352c3dd-8c2e-408c-bcb2-b69e784f5a77 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Created volume successfully. >2018-08-23 10:14:01.962 257256 INFO cinder.volume.manager [req-80098e1d-2dd2-41e3-8135-a2c430d84266 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Create snapshot completed successfully >2018-08-23 10:14:13.677 257256 INFO cinder.volume.manager [req-a5a11f4b-8641-4cda-9383-fe3eb5e97f99 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Create snapshot completed successfully >2018-08-23 10:14:21.351 257256 INFO cinder.volume.manager [req-14b320a3-f371-44f3-8717-e9d6bdb5d14a 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Create snapshot completed successfully >2018-08-23 10:14:27.714 257256 INFO cinder.volume.manager [req-86d973aa-25b2-4ed7-bc43-a1da25e4fa50 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Delete snapshot completed successfully. >2018-08-23 10:14:27.991 257256 INFO cinder.volume.manager [req-63e96cbc-4479-44d9-bedc-3de8e7388aeb 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Delete snapshot completed successfully. >2018-08-23 10:14:28.218 257256 INFO cinder.volume.manager [req-edca1d96-b76a-48d5-aac2-fdefdc32ce7b 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Delete snapshot completed successfully. >2018-08-23 10:14:30.110 257256 INFO cinder.volume.manager [req-7d05edbf-15de-4f92-ba50-924274da3be5 14d073d143674488b0c1d1ab55f745de aad7f0103c814d4d80d560d5d6c93fee - default default] Deleted volume successfully. >2018-08-23 10:14:35.503 257256 INFO cinder.volume.flows.manager.create_volume [req-6efd4232-dcf8-4bdf-b050-79ab3bdbccae ede5f9c95ba64032809193b7b244e514 821ebc93433048dba667f72c591a7574 - default default] Volume 615cfce0-a0f6-4eb5-8f7c-a036bf9c5c6c: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-615cfce0-a0f6-4eb5-8f7c-a036bf9c5c6c'} >2018-08-23 10:14:36.027 257256 INFO cinder.volume.flows.manager.create_volume [req-6efd4232-dcf8-4bdf-b050-79ab3bdbccae ede5f9c95ba64032809193b7b244e514 821ebc93433048dba667f72c591a7574 - default default] Volume volume-615cfce0-a0f6-4eb5-8f7c-a036bf9c5c6c (615cfce0-a0f6-4eb5-8f7c-a036bf9c5c6c): created successfully >2018-08-23 10:14:36.031 257256 INFO cinder.volume.manager [req-6efd4232-dcf8-4bdf-b050-79ab3bdbccae ede5f9c95ba64032809193b7b244e514 821ebc93433048dba667f72c591a7574 - default default] Created volume successfully. >2018-08-23 10:14:43.930 257256 INFO cinder.volume.manager [req-0f134fc9-60ad-4198-8971-8d5c5db3aed3 ede5f9c95ba64032809193b7b244e514 821ebc93433048dba667f72c591a7574 - default default] Create snapshot completed successfully >2018-08-23 10:14:47.086 257256 INFO cinder.volume.manager [req-bf0cb2f2-bf44-4280-a76f-9b2dc6c6a109 ede5f9c95ba64032809193b7b244e514 821ebc93433048dba667f72c591a7574 - default default] Delete snapshot completed successfully. >2018-08-23 10:14:48.760 257256 INFO cinder.volume.manager [req-f758bc9d-78d7-4309-91ca-f4866ddde164 ede5f9c95ba64032809193b7b244e514 821ebc93433048dba667f72c591a7574 - default default] Deleted volume successfully. >2018-08-23 10:15:10.920 257256 INFO cinder.volume.flows.manager.create_volume [req-4d5c4b6e-7193-46fe-897e-d2cb34ad42a8 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume 944b62f6-925e-4e4c-b628-f110de0f2c01: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-944b62f6-925e-4e4c-b628-f110de0f2c01', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab541849d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-23 10:15:12.300 257256 INFO cinder.image.image_utils [req-4d5c4b6e-7193-46fe-897e-d2cb34ad42a8 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Image download 12.00 MB at 8.71 MB/s >2018-08-23 10:15:21.845 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-23 10:15:28.478 257256 INFO cinder.image.image_utils [req-4d5c4b6e-7193-46fe-897e-d2cb34ad42a8 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Converted 39.00 MB image at 16.10 MB/s >2018-08-23 10:15:30.530 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-4d5c4b6e-7193-46fe-897e-d2cb34ad42a8 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Need to remove FC Zone, building initiator target map. >2018-08-23 10:15:30.886 257256 INFO cinder.volume.flows.manager.create_volume [req-4d5c4b6e-7193-46fe-897e-d2cb34ad42a8 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume volume-944b62f6-925e-4e4c-b628-f110de0f2c01 (944b62f6-925e-4e4c-b628-f110de0f2c01): created successfully >2018-08-23 10:15:30.890 257256 INFO cinder.volume.manager [req-4d5c4b6e-7193-46fe-897e-d2cb34ad42a8 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Created volume successfully. >2018-08-23 10:15:31.748 257256 INFO cinder.volume.flows.manager.create_volume [req-eb7332fd-35b8-4465-ae13-81ab62a4b9a1 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume b3512758-50d7-4f80-99d3-9207391a6808: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b3512758-50d7-4f80-99d3-9207391a6808', 'source_volid': '944b62f6-925e-4e4c-b628-f110de0f2c01', 'source_volstatus': u'available'} >2018-08-23 10:15:38.425 257256 INFO cinder.volume.flows.manager.create_volume [req-eb7332fd-35b8-4465-ae13-81ab62a4b9a1 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume volume-b3512758-50d7-4f80-99d3-9207391a6808 (b3512758-50d7-4f80-99d3-9207391a6808): created successfully >2018-08-23 10:15:38.429 257256 INFO cinder.volume.manager [req-eb7332fd-35b8-4465-ae13-81ab62a4b9a1 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Created volume successfully. >2018-08-23 10:15:39.066 257256 INFO cinder.volume.flows.manager.create_volume [req-9bd8ccd5-6582-40df-9e54-98fb9f517eab 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume 1604ac3c-de97-45e4-835d-8955f2398e3b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1604ac3c-de97-45e4-835d-8955f2398e3b'} >2018-08-23 10:15:39.593 257256 INFO cinder.volume.flows.manager.create_volume [req-9bd8ccd5-6582-40df-9e54-98fb9f517eab 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume volume-1604ac3c-de97-45e4-835d-8955f2398e3b (1604ac3c-de97-45e4-835d-8955f2398e3b): created successfully >2018-08-23 10:15:39.597 257256 INFO cinder.volume.manager [req-9bd8ccd5-6582-40df-9e54-98fb9f517eab 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Created volume successfully. >2018-08-23 10:15:40.391 257256 INFO cinder.volume.flows.manager.create_volume [req-44e04b91-3d4f-46b0-b667-eab16ee66228 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume 257ea091-b527-43a5-aecb-f924ed1d0669: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-257ea091-b527-43a5-aecb-f924ed1d0669', 'source_volid': '1604ac3c-de97-45e4-835d-8955f2398e3b', 'source_volstatus': u'available'} >2018-08-23 10:16:30.662 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-44e04b91-3d4f-46b0-b667-eab16ee66228 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Unable to get remote copy information for volume volume-257ea091-b527-43a5-aecb-f924ed1d0669 >2018-08-23 10:16:30.935 257256 INFO cinder.volume.flows.manager.create_volume [req-44e04b91-3d4f-46b0-b667-eab16ee66228 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Volume volume-257ea091-b527-43a5-aecb-f924ed1d0669 (257ea091-b527-43a5-aecb-f924ed1d0669): created successfully >2018-08-23 10:16:30.939 257256 INFO cinder.volume.manager [req-44e04b91-3d4f-46b0-b667-eab16ee66228 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Created volume successfully. >2018-08-23 10:16:34.320 257256 INFO cinder.volume.manager [req-63343b38-5b32-4dbe-bc8b-f8e212aa9926 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Deleted volume successfully. >2018-08-23 10:16:34.790 257256 INFO cinder.volume.manager [req-f36eaad3-b8e0-4072-9f17-aabf0da753ba 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Deleted volume successfully. >2018-08-23 10:16:34.816 257256 INFO cinder.volume.manager [req-d58ad64e-0a88-482d-95b9-359bbbd55452 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Deleted volume successfully. >2018-08-23 10:16:35.037 257256 INFO cinder.volume.manager [req-c64b57af-b75e-43b6-af2f-b20f5032a701 6c4f232e6d5041b689e15a8a04c7c305 05acc6574e5a4514b5099eab3856e8fb - default default] Deleted volume successfully. >2018-08-23 10:16:39.471 257256 INFO cinder.volume.flows.manager.create_volume [req-0b8a03e6-25ce-4baf-8662-dafb5855e6b7 60d3786328424868ac0d6f41d42091c4 2e8f3c452a8c42d9aa006f413165527d - default default] Volume 20b65793-0917-4338-8e64-88cd3ea00b1c: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-20b65793-0917-4338-8e64-88cd3ea00b1c'} >2018-08-23 10:16:40.020 257256 INFO cinder.volume.flows.manager.create_volume [req-0b8a03e6-25ce-4baf-8662-dafb5855e6b7 60d3786328424868ac0d6f41d42091c4 2e8f3c452a8c42d9aa006f413165527d - default default] Volume volume-20b65793-0917-4338-8e64-88cd3ea00b1c (20b65793-0917-4338-8e64-88cd3ea00b1c): created successfully >2018-08-23 10:16:40.024 257256 INFO cinder.volume.manager [req-0b8a03e6-25ce-4baf-8662-dafb5855e6b7 60d3786328424868ac0d6f41d42091c4 2e8f3c452a8c42d9aa006f413165527d - default default] Created volume successfully. >2018-08-23 10:16:42.712 257256 INFO cinder.volume.manager [req-a613ecea-8943-45d2-98aa-2e266d2d09e9 60d3786328424868ac0d6f41d42091c4 2e8f3c452a8c42d9aa006f413165527d - default default] Deleted volume successfully. >2018-08-23 10:17:06.678 257256 INFO cinder.volume.flows.manager.create_volume [req-04d418e5-812c-492b-9823-13411644a601 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] Volume 5e1aa4be-66bf-4365-9ec4-45e232dfb0de: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5e1aa4be-66bf-4365-9ec4-45e232dfb0de'} >2018-08-23 10:17:07.506 257256 INFO cinder.volume.flows.manager.create_volume [req-04d418e5-812c-492b-9823-13411644a601 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] Volume volume-5e1aa4be-66bf-4365-9ec4-45e232dfb0de (5e1aa4be-66bf-4365-9ec4-45e232dfb0de): created successfully >2018-08-23 10:17:07.510 257256 INFO cinder.volume.manager [req-04d418e5-812c-492b-9823-13411644a601 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] Created volume successfully. >2018-08-23 10:17:10.163 257256 INFO cinder.volume.manager [req-cf53ed23-b0b7-4850-a1a1-d9d391a46253 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] Deleted volume successfully. >2018-08-23 10:17:10.617 257256 INFO cinder.volume.flows.manager.create_volume [req-2c658636-1ca0-4112-9935-3d5b92959611 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] Volume 9737f262-95c8-4fcd-9def-efc65764bf10: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9737f262-95c8-4fcd-9def-efc65764bf10'} >2018-08-23 10:17:11.503 257256 INFO cinder.volume.flows.manager.create_volume [req-2c658636-1ca0-4112-9935-3d5b92959611 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] Volume volume-9737f262-95c8-4fcd-9def-efc65764bf10 (9737f262-95c8-4fcd-9def-efc65764bf10): created successfully >2018-08-23 10:17:11.507 257256 INFO cinder.volume.manager [req-2c658636-1ca0-4112-9935-3d5b92959611 9b87fcbc69be433b967a93bcd6258ab8 176f5f1c573648639325fa56e18c3bb2 - default default] Created volume successfully. >2018-08-23 10:17:13.184 257256 INFO cinder.volume.manager [req-9a4fa407-efa0-469c-a954-8573745142a0 7f5e563847404ce5874d0f939b5c7e7d 7ed8a9ba820e42ae8986f1268e376e2d - default default] Transfer volume completed successfully. >2018-08-23 10:17:19.596 257256 INFO cinder.volume.manager [req-e747d941-da10-4331-a5fa-aaa1e7d21973 de56463127c44e28b164ac50898f9ff9 e82fdb2151394bcc8f3b6f45d708d072 - default default] Deleted volume successfully. >2018-08-23 10:17:28.880 257256 INFO cinder.volume.flows.manager.create_volume [req-eac7bf5b-8841-4d2a-a92a-cef065ca8bed 3c36f1a8fa0348a395c81915fcd4d691 b068e900ea1742b3985c951d04979c5f - default default] Volume dc86f45e-977d-4c9d-b891-97d218464aa4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-dc86f45e-977d-4c9d-b891-97d218464aa4'} >2018-08-23 10:17:29.800 257256 INFO cinder.volume.flows.manager.create_volume [req-eac7bf5b-8841-4d2a-a92a-cef065ca8bed 3c36f1a8fa0348a395c81915fcd4d691 b068e900ea1742b3985c951d04979c5f - default default] Volume volume-dc86f45e-977d-4c9d-b891-97d218464aa4 (dc86f45e-977d-4c9d-b891-97d218464aa4): created successfully >2018-08-23 10:17:29.805 257256 INFO cinder.volume.manager [req-eac7bf5b-8841-4d2a-a92a-cef065ca8bed 3c36f1a8fa0348a395c81915fcd4d691 b068e900ea1742b3985c951d04979c5f - default default] Created volume successfully. >2018-08-23 10:17:37.222 257256 INFO cinder.volume.manager [req-7fff3f17-9c55-4613-98bd-b66a2812352e 3c36f1a8fa0348a395c81915fcd4d691 b068e900ea1742b3985c951d04979c5f - default default] Deleted volume successfully. >2018-08-23 10:17:44.689 257256 INFO cinder.volume.flows.manager.create_volume [req-07397530-023e-4c16-ac01-b1914a522a76 5c4744c4b34147d29f06c7c7f7c4da94 ac8ee7b40aeb4edb81fd25693383a764 - default default] Volume 6d73a684-f3c3-45e8-b8ae-641eaa520e6e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6d73a684-f3c3-45e8-b8ae-641eaa520e6e'} >2018-08-23 10:17:45.718 257256 INFO cinder.volume.flows.manager.create_volume [req-07397530-023e-4c16-ac01-b1914a522a76 5c4744c4b34147d29f06c7c7f7c4da94 ac8ee7b40aeb4edb81fd25693383a764 - default default] Volume volume-6d73a684-f3c3-45e8-b8ae-641eaa520e6e (6d73a684-f3c3-45e8-b8ae-641eaa520e6e): created successfully >2018-08-23 10:17:45.723 257256 INFO cinder.volume.manager [req-07397530-023e-4c16-ac01-b1914a522a76 5c4744c4b34147d29f06c7c7f7c4da94 ac8ee7b40aeb4edb81fd25693383a764 - default default] Created volume successfully. >2018-08-23 10:17:54.317 257256 INFO cinder.volume.manager [req-9d0931b7-7827-46b1-ad5c-f3a72f59b057 5c4744c4b34147d29f06c7c7f7c4da94 ac8ee7b40aeb4edb81fd25693383a764 - default default] Create snapshot completed successfully >2018-08-23 10:17:58.509 257256 INFO cinder.volume.manager [req-58ba9960-42fe-4781-8265-dfc83d8bbb8f 5c4744c4b34147d29f06c7c7f7c4da94 ac8ee7b40aeb4edb81fd25693383a764 - default default] Delete snapshot completed successfully. >2018-08-23 10:18:00.792 257256 INFO cinder.volume.manager [req-fe10447d-38c1-4f77-af21-8ebc0dbf35ee 5c4744c4b34147d29f06c7c7f7c4da94 ac8ee7b40aeb4edb81fd25693383a764 - default default] Deleted volume successfully. >2018-08-24 03:29:35.441 257256 INFO cinder.volume.flows.manager.create_volume [req-f2fe3ce6-c4f2-4cfa-9516-af3e96b08f8f e1afb1a5a7a24425937e1c1cfbacc2d1 19520f8e5d04458f97661c84c400de77 - default default] Volume 5ab1f8d0-6f30-430d-a748-f3f0f62b06c7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5ab1f8d0-6f30-430d-a748-f3f0f62b06c7'} >2018-08-24 03:29:35.986 257256 INFO cinder.volume.flows.manager.create_volume [req-f2fe3ce6-c4f2-4cfa-9516-af3e96b08f8f e1afb1a5a7a24425937e1c1cfbacc2d1 19520f8e5d04458f97661c84c400de77 - default default] Volume volume-5ab1f8d0-6f30-430d-a748-f3f0f62b06c7 (5ab1f8d0-6f30-430d-a748-f3f0f62b06c7): created successfully >2018-08-24 03:29:35.990 257256 INFO cinder.volume.manager [req-f2fe3ce6-c4f2-4cfa-9516-af3e96b08f8f e1afb1a5a7a24425937e1c1cfbacc2d1 19520f8e5d04458f97661c84c400de77 - default default] Created volume successfully. >2018-08-24 03:29:39.513 257256 INFO cinder.volume.manager [req-48e39118-37a6-436d-b338-bcd53527b885 e1afb1a5a7a24425937e1c1cfbacc2d1 19520f8e5d04458f97661c84c400de77 - default default] Deleted volume successfully. >2018-08-24 03:29:48.998 257256 INFO cinder.volume.flows.manager.create_volume [req-f23c8fe6-e5e6-4a8f-8156-e389aae190d3 0e702baf760d4116844b8bbe0cf97d81 1a5c1148c88b497db9ac33a448e90724 - default default] Volume 1c054cef-e7b7-4bb3-9902-9ba5a4de9089: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1c054cef-e7b7-4bb3-9902-9ba5a4de9089'} >2018-08-24 03:29:49.632 257256 INFO cinder.volume.flows.manager.create_volume [req-f23c8fe6-e5e6-4a8f-8156-e389aae190d3 0e702baf760d4116844b8bbe0cf97d81 1a5c1148c88b497db9ac33a448e90724 - default default] Volume volume-1c054cef-e7b7-4bb3-9902-9ba5a4de9089 (1c054cef-e7b7-4bb3-9902-9ba5a4de9089): created successfully >2018-08-24 03:29:49.636 257256 INFO cinder.volume.manager [req-f23c8fe6-e5e6-4a8f-8156-e389aae190d3 0e702baf760d4116844b8bbe0cf97d81 1a5c1148c88b497db9ac33a448e90724 - default default] Created volume successfully. >2018-08-24 03:29:52.705 257256 WARNING cinder.quota [req-0372fc5b-9bd0-4ad4-8120-288cd6effd53 0e702baf760d4116844b8bbe0cf97d81 1a5c1148c88b497db9ac33a448e90724 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1588470265 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1588470265, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 03:29:52.706 257256 WARNING cinder.quota [req-0372fc5b-9bd0-4ad4-8120-288cd6effd53 0e702baf760d4116844b8bbe0cf97d81 1a5c1148c88b497db9ac33a448e90724 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1588470265 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1588470265, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 03:29:52.965 257256 INFO cinder.volume.manager [req-0372fc5b-9bd0-4ad4-8120-288cd6effd53 0e702baf760d4116844b8bbe0cf97d81 1a5c1148c88b497db9ac33a448e90724 - default default] Deleted volume successfully. >2018-08-24 03:30:01.801 257256 INFO cinder.volume.flows.manager.create_volume [req-5d4991f6-665e-4c8c-887f-91f3b802deb8 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] Volume 199522bc-935c-4e7d-ab60-4517547bfecd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-199522bc-935c-4e7d-ab60-4517547bfecd'} >2018-08-24 03:30:02.391 257256 INFO cinder.volume.flows.manager.create_volume [req-5d4991f6-665e-4c8c-887f-91f3b802deb8 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] Volume volume-199522bc-935c-4e7d-ab60-4517547bfecd (199522bc-935c-4e7d-ab60-4517547bfecd): created successfully >2018-08-24 03:30:02.394 257256 INFO cinder.volume.manager [req-5d4991f6-665e-4c8c-887f-91f3b802deb8 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] Created volume successfully. >2018-08-24 03:30:03.955 257256 INFO cinder.volume.manager [req-3bbb7cae-e33a-4725-9b7e-854bf7057257 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] Retype volume completed successfully. >2018-08-24 03:30:07.968 257256 WARNING cinder.quota [req-a08f0c0f-7a52-405f-9955-961d74004c81 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1517942536 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1517942536, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 03:30:07.969 257256 WARNING cinder.quota [req-a08f0c0f-7a52-405f-9955-961d74004c81 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1517942536 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1517942536, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 03:30:08.218 257256 INFO cinder.volume.manager [req-a08f0c0f-7a52-405f-9955-961d74004c81 dca1f8d8ea6e47669e188be404620758 f7c0a20975b64dc292875280479983c7 - default default] Deleted volume successfully. >2018-08-24 03:30:50.112 257256 INFO cinder.volume.flows.manager.create_volume [req-088fadbb-6847-4819-bd00-b921fd170934 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume c0bfef51-636d-4f57-9d9a-dbdd96aa5a22: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c0bfef51-636d-4f57-9d9a-dbdd96aa5a22'} >2018-08-24 03:30:50.935 257256 INFO cinder.volume.flows.manager.create_volume [req-088fadbb-6847-4819-bd00-b921fd170934 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume volume-c0bfef51-636d-4f57-9d9a-dbdd96aa5a22 (c0bfef51-636d-4f57-9d9a-dbdd96aa5a22): created successfully >2018-08-24 03:30:50.938 257256 INFO cinder.volume.manager [req-088fadbb-6847-4819-bd00-b921fd170934 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Created volume successfully. >2018-08-24 03:30:51.836 257256 INFO cinder.volume.manager [req-cb90f259-b5e4-47eb-b483-8397928b3eba ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Attaching volume c0bfef51-636d-4f57-9d9a-dbdd96aa5a22 to instance f60cf37d-c850-4f3c-b02c-84efaa8013df at mountpoint /dev/vdb on host None. >2018-08-24 03:30:51.907 257256 INFO cinder.volume.manager [req-cb90f259-b5e4-47eb-b483-8397928b3eba ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Attach volume completed successfully. >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager [req-6f118266-b2ff-40b4-8c51-3a3348552218 b12b802705574e5f9f728ec46d9ffe9b f7b0d351ffb6488a97bfa0b9c69b2dc9 - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager force=force) >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-24 03:30:53.466 257256 ERROR cinder.volume.manager >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server [req-6f118266-b2ff-40b4-8c51-3a3348552218 b12b802705574e5f9f728ec46d9ffe9b f7b0d351ffb6488a97bfa0b9c69b2dc9 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-24 03:30:53.468 257256 ERROR oslo_messaging.rpc.server >2018-08-24 03:30:53.796 257256 INFO cinder.volume.manager [req-46eae6de-7050-4de8-ba8e-d14598a7da07 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Detaching volume c0bfef51-636d-4f57-9d9a-dbdd96aa5a22 from instance f60cf37d-c850-4f3c-b02c-84efaa8013df. >2018-08-24 03:30:53.879 257256 INFO cinder.volume.manager [req-46eae6de-7050-4de8-ba8e-d14598a7da07 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Detach volume completed successfully. >2018-08-24 03:30:55.975 257031 INFO cinder.volume.flows.manager.create_volume [req-3134ef1f-fab3-43d5-a0f3-e9aced58d75a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume 4dabe68a-797c-4ec9-bb00-f967d28045c4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4dabe68a-797c-4ec9-bb00-f967d28045c4'} >2018-08-24 03:30:56.885 257031 INFO cinder.volume.flows.manager.create_volume [req-3134ef1f-fab3-43d5-a0f3-e9aced58d75a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume volume-4dabe68a-797c-4ec9-bb00-f967d28045c4 (4dabe68a-797c-4ec9-bb00-f967d28045c4): created successfully >2018-08-24 03:30:56.888 257031 INFO cinder.volume.manager [req-3134ef1f-fab3-43d5-a0f3-e9aced58d75a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Created volume successfully. >2018-08-24 03:30:59.547 257256 INFO cinder.volume.flows.manager.create_volume [req-005af8ac-6d25-40d9-ba73-c791bca52e19 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume c560b1bb-9fb3-4152-a311-44f0170a1aa0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c560b1bb-9fb3-4152-a311-44f0170a1aa0'} >2018-08-24 03:30:59.575 257031 INFO cinder.volume.manager [req-f6644645-4ef0-499c-9d3a-1d1cb0fb0d0c b12b802705574e5f9f728ec46d9ffe9b f7b0d351ffb6488a97bfa0b9c69b2dc9 - default default] Deleted volume successfully. >2018-08-24 03:31:00.345 257256 INFO cinder.volume.flows.manager.create_volume [req-005af8ac-6d25-40d9-ba73-c791bca52e19 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume volume-c560b1bb-9fb3-4152-a311-44f0170a1aa0 (c560b1bb-9fb3-4152-a311-44f0170a1aa0): created successfully >2018-08-24 03:31:00.349 257256 INFO cinder.volume.manager [req-005af8ac-6d25-40d9-ba73-c791bca52e19 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Created volume successfully. >2018-08-24 03:31:02.837 257256 INFO cinder.volume.manager [req-40fb254c-dfee-4f72-978c-6e74803a476b b12b802705574e5f9f728ec46d9ffe9b f7b0d351ffb6488a97bfa0b9c69b2dc9 - default default] Deleted volume successfully. >2018-08-24 03:31:03.132 257256 INFO cinder.volume.flows.manager.create_volume [req-f3304e3c-63e1-45a4-b9cc-dfd3afb0794a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume 62a882b4-f4b8-4b36-8c77-9a42e8a98f0b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-62a882b4-f4b8-4b36-8c77-9a42e8a98f0b'} >2018-08-24 03:31:04.030 257256 INFO cinder.volume.flows.manager.create_volume [req-f3304e3c-63e1-45a4-b9cc-dfd3afb0794a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume volume-62a882b4-f4b8-4b36-8c77-9a42e8a98f0b (62a882b4-f4b8-4b36-8c77-9a42e8a98f0b): created successfully >2018-08-24 03:31:04.034 257256 INFO cinder.volume.manager [req-f3304e3c-63e1-45a4-b9cc-dfd3afb0794a ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Created volume successfully. >2018-08-24 03:31:05.641 257256 INFO cinder.volume.manager [req-928f25f2-2e83-4706-be4e-dc57cdd7e812 b12b802705574e5f9f728ec46d9ffe9b f7b0d351ffb6488a97bfa0b9c69b2dc9 - default default] Deleted volume successfully. >2018-08-24 03:31:06.037 257256 INFO cinder.volume.flows.manager.create_volume [req-b8e7a3c7-6849-4e50-b165-9750d0a81557 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume 0fd92884-b68c-42eb-a4b8-3ab282378201: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0fd92884-b68c-42eb-a4b8-3ab282378201'} >2018-08-24 03:31:06.560 257256 INFO cinder.volume.flows.manager.create_volume [req-b8e7a3c7-6849-4e50-b165-9750d0a81557 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume volume-0fd92884-b68c-42eb-a4b8-3ab282378201 (0fd92884-b68c-42eb-a4b8-3ab282378201): created successfully >2018-08-24 03:31:06.564 257256 INFO cinder.volume.manager [req-b8e7a3c7-6849-4e50-b165-9750d0a81557 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Created volume successfully. >2018-08-24 03:31:08.587 257256 INFO cinder.volume.manager [req-5cf532b6-cfc3-40ae-8567-71af7507fcd2 b12b802705574e5f9f728ec46d9ffe9b f7b0d351ffb6488a97bfa0b9c69b2dc9 - default default] Deleted volume successfully. >2018-08-24 03:31:08.930 257256 INFO cinder.volume.flows.manager.create_volume [req-46cca44e-52b9-4331-a41b-9d0fdb54f669 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume bab79637-213c-4615-bb66-0b5327fdf35b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bab79637-213c-4615-bb66-0b5327fdf35b'} >2018-08-24 03:31:09.456 257256 INFO cinder.volume.flows.manager.create_volume [req-46cca44e-52b9-4331-a41b-9d0fdb54f669 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Volume volume-bab79637-213c-4615-bb66-0b5327fdf35b (bab79637-213c-4615-bb66-0b5327fdf35b): created successfully >2018-08-24 03:31:09.460 257256 INFO cinder.volume.manager [req-46cca44e-52b9-4331-a41b-9d0fdb54f669 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Created volume successfully. >2018-08-24 03:31:15.951 257256 INFO cinder.volume.manager [req-0bb5d4a8-34fb-4d8e-8a73-28f24e6df0a0 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Deleted volume successfully. >2018-08-24 03:31:16.209 257256 INFO cinder.volume.manager [req-7a023f6e-54a8-4213-86d2-6ba96e8e5be2 ad0f1ae3fccd4480b162e87e03a7f957 5225151c89404f888a6662507be573d7 - default default] Deleted volume successfully. >2018-08-24 03:31:22.858 257256 INFO cinder.volume.flows.manager.create_volume [req-3bb6fc6a-392b-4802-8cd9-383adf6ccb67 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Volume a00a4723-5268-4941-979b-90892da0137f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a00a4723-5268-4941-979b-90892da0137f'} >2018-08-24 03:31:23.379 257256 INFO cinder.volume.flows.manager.create_volume [req-3bb6fc6a-392b-4802-8cd9-383adf6ccb67 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Volume volume-a00a4723-5268-4941-979b-90892da0137f (a00a4723-5268-4941-979b-90892da0137f): created successfully >2018-08-24 03:31:23.383 257256 INFO cinder.volume.manager [req-3bb6fc6a-392b-4802-8cd9-383adf6ccb67 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Created volume successfully. >2018-08-24 03:31:25.362 257256 INFO cinder.volume.flows.manager.create_volume [req-fe796d5c-0ee9-4895-97fb-c1b81161b1ff 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Volume 80e04951-c241-44fc-85a3-848b00f376f9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-80e04951-c241-44fc-85a3-848b00f376f9'} >2018-08-24 03:31:25.961 257256 INFO cinder.volume.flows.manager.create_volume [req-fe796d5c-0ee9-4895-97fb-c1b81161b1ff 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Volume volume-80e04951-c241-44fc-85a3-848b00f376f9 (80e04951-c241-44fc-85a3-848b00f376f9): created successfully >2018-08-24 03:31:25.964 257256 INFO cinder.volume.manager [req-fe796d5c-0ee9-4895-97fb-c1b81161b1ff 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Created volume successfully. >2018-08-24 03:31:27.480 257031 INFO cinder.volume.flows.manager.create_volume [req-1c85adf9-62b0-4d38-87e4-a99e0d9aa49a 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Volume f8235a47-ffef-4558-aab4-729191d305c6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f8235a47-ffef-4558-aab4-729191d305c6'} >2018-08-24 03:31:28.003 257031 INFO cinder.volume.flows.manager.create_volume [req-1c85adf9-62b0-4d38-87e4-a99e0d9aa49a 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Volume volume-f8235a47-ffef-4558-aab4-729191d305c6 (f8235a47-ffef-4558-aab4-729191d305c6): created successfully >2018-08-24 03:31:28.007 257031 INFO cinder.volume.manager [req-1c85adf9-62b0-4d38-87e4-a99e0d9aa49a 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Created volume successfully. >2018-08-24 03:31:29.305 257256 INFO cinder.volume.flows.manager.create_volume [req-95471159-7236-4c5f-87c6-f01d747cea2b ec21e035b091488f80130d0948aaecf3 cad135d5b5f44a2ba0e2f4b075553aec - default default] Volume 1d65faf2-4d36-4d7d-9a8e-75a4fc8c3cb9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1d65faf2-4d36-4d7d-9a8e-75a4fc8c3cb9'} >2018-08-24 03:31:32.777 257256 INFO cinder.volume.flows.manager.create_volume [req-95471159-7236-4c5f-87c6-f01d747cea2b ec21e035b091488f80130d0948aaecf3 cad135d5b5f44a2ba0e2f4b075553aec - default default] Volume volume-1d65faf2-4d36-4d7d-9a8e-75a4fc8c3cb9 (1d65faf2-4d36-4d7d-9a8e-75a4fc8c3cb9): created successfully >2018-08-24 03:31:32.780 257256 INFO cinder.volume.manager [req-95471159-7236-4c5f-87c6-f01d747cea2b ec21e035b091488f80130d0948aaecf3 cad135d5b5f44a2ba0e2f4b075553aec - default default] Created volume successfully. >2018-08-24 03:31:36.816 257256 INFO cinder.volume.manager [req-72c8b25b-e541-4a15-940c-342adce1d458 ec21e035b091488f80130d0948aaecf3 cad135d5b5f44a2ba0e2f4b075553aec - default default] Deleted volume successfully. >2018-08-24 03:31:37.066 257256 INFO cinder.volume.manager [req-3140d1c1-34f2-47a3-8172-27709a591c1d 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Deleted volume successfully. >2018-08-24 03:31:37.088 257256 INFO cinder.volume.manager [req-4b41949d-b268-44fe-8ce3-a45f114ec12a 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Deleted volume successfully. >2018-08-24 03:31:37.301 257031 INFO cinder.volume.manager [req-f1d8a858-7ee0-4004-969c-9df9a8628583 36341e106c804b649431611adb192546 8d5cab7e48124e0b9d467bba68da964d - default default] Deleted volume successfully. >2018-08-24 03:31:43.114 257256 INFO cinder.volume.flows.manager.create_volume [req-95cdfadd-2c7c-4911-b23f-cc61fed79491 8aa8181692e044e6a66bdc424269f545 262a3967f25c4cb19b234498a9ab1c88 - default default] Volume 064109b5-79c9-4588-a64f-d345b05d08be: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-064109b5-79c9-4588-a64f-d345b05d08be'} >2018-08-24 03:31:43.637 257256 INFO cinder.volume.flows.manager.create_volume [req-95cdfadd-2c7c-4911-b23f-cc61fed79491 8aa8181692e044e6a66bdc424269f545 262a3967f25c4cb19b234498a9ab1c88 - default default] Volume volume-064109b5-79c9-4588-a64f-d345b05d08be (064109b5-79c9-4588-a64f-d345b05d08be): created successfully >2018-08-24 03:31:43.641 257256 INFO cinder.volume.manager [req-95cdfadd-2c7c-4911-b23f-cc61fed79491 8aa8181692e044e6a66bdc424269f545 262a3967f25c4cb19b234498a9ab1c88 - default default] Created volume successfully. >2018-08-24 03:31:47.787 257256 INFO cinder.volume.manager [req-566fd1be-5391-4242-8dc7-8b833a5e14a7 8aa8181692e044e6a66bdc424269f545 262a3967f25c4cb19b234498a9ab1c88 - default default] Deleted volume successfully. >2018-08-24 03:31:55.217 257256 INFO cinder.volume.flows.manager.create_volume [req-6370d416-1a23-49a8-9814-cd5b53685a78 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] Volume 2008f328-02d6-46c0-875a-7b0aed0b90db: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2008f328-02d6-46c0-875a-7b0aed0b90db'} >2018-08-24 03:31:55.740 257256 INFO cinder.volume.flows.manager.create_volume [req-6370d416-1a23-49a8-9814-cd5b53685a78 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] Volume volume-2008f328-02d6-46c0-875a-7b0aed0b90db (2008f328-02d6-46c0-875a-7b0aed0b90db): created successfully >2018-08-24 03:31:55.744 257256 INFO cinder.volume.manager [req-6370d416-1a23-49a8-9814-cd5b53685a78 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] Created volume successfully. >2018-08-24 03:31:57.908 257256 INFO cinder.volume.manager [req-30cbdb15-7acc-47c6-8186-243ee9799142 d926bff95bf144fe8d12d13db2cdcd7e 3416eeb88d5047b8867cb12b8c4d7a0e - default default] Transfer volume completed successfully. >2018-08-24 03:31:59.987 257256 INFO cinder.volume.manager [req-7103ce8a-cf1b-47df-977c-c1d202586217 7c27e6989e5049b49a9c05ca57c748ba 8b94e270dd2a43bf863b59b938513107 - default default] Deleted volume successfully. >2018-08-24 03:32:01.118 257256 INFO cinder.volume.flows.manager.create_volume [req-37fcabb1-26c8-4985-9baf-af69bcf73835 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] Volume 8a60096a-d09d-48cb-a7e4-00e1996af464: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8a60096a-d09d-48cb-a7e4-00e1996af464'} >2018-08-24 03:32:01.649 257256 INFO cinder.volume.flows.manager.create_volume [req-37fcabb1-26c8-4985-9baf-af69bcf73835 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] Volume volume-8a60096a-d09d-48cb-a7e4-00e1996af464 (8a60096a-d09d-48cb-a7e4-00e1996af464): created successfully >2018-08-24 03:32:01.653 257256 INFO cinder.volume.manager [req-37fcabb1-26c8-4985-9baf-af69bcf73835 a824c89cb6474b46a6e6f3e5e2724c9d 17494d712bff4c0596c2daa5bc4c32df - default default] Created volume successfully. >2018-08-24 03:32:04.808 257256 INFO cinder.volume.manager [req-ffbd66f8-8cc1-4051-aaf2-755183669e43 7c27e6989e5049b49a9c05ca57c748ba 8b94e270dd2a43bf863b59b938513107 - default default] Deleted volume successfully. >2018-08-24 03:32:11.329 257256 INFO cinder.volume.flows.manager.create_volume [req-2e4c1acf-425e-4cbc-80f0-6ddd8da9ef48 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Volume 8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a'} >2018-08-24 03:32:11.855 257256 INFO cinder.volume.flows.manager.create_volume [req-2e4c1acf-425e-4cbc-80f0-6ddd8da9ef48 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Volume volume-8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a (8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a): created successfully >2018-08-24 03:32:11.859 257256 INFO cinder.volume.manager [req-2e4c1acf-425e-4cbc-80f0-6ddd8da9ef48 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Created volume successfully. >2018-08-24 03:32:20.194 257256 INFO cinder.volume.manager [req-faaf9556-525f-4fb8-bc7a-c794530b5ad6 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Attaching volume 8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a to instance 082ee0b4-e1a1-403e-8aee-3d42102a1623 at mountpoint /dev/vdb on host None. >2018-08-24 03:32:20.272 257256 INFO cinder.volume.manager [req-faaf9556-525f-4fb8-bc7a-c794530b5ad6 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Attach volume completed successfully. >2018-08-24 03:32:20.927 257256 INFO cinder.volume.manager [req-33ec486e-9381-4fd1-93df-a3e117b6853f 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Detaching volume 8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a from instance 082ee0b4-e1a1-403e-8aee-3d42102a1623. >2018-08-24 03:32:21.020 257256 INFO cinder.volume.manager [req-33ec486e-9381-4fd1-93df-a3e117b6853f 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Detach volume completed successfully. >2018-08-24 03:32:27.302 257256 INFO cinder.volume.manager [req-485ff626-d4d6-4f7e-a139-893f38d70666 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Attaching volume 8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a to instance c8eadb41-5f54-4fc0-8e0a-c0d3b73334df at mountpoint /dev/vdb on host None. >2018-08-24 03:32:27.381 257256 INFO cinder.volume.manager [req-485ff626-d4d6-4f7e-a139-893f38d70666 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Attach volume completed successfully. >2018-08-24 03:32:27.667 257256 INFO cinder.volume.manager [req-bd733f6e-eed2-445f-a43e-182169cb168a 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Detaching volume 8aeda035-e2ab-40cf-ad0a-51d5a2a08e9a from instance c8eadb41-5f54-4fc0-8e0a-c0d3b73334df. >2018-08-24 03:32:27.755 257256 INFO cinder.volume.manager [req-bd733f6e-eed2-445f-a43e-182169cb168a 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Detach volume completed successfully. >2018-08-24 03:32:39.901 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 03:32:58.334 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-19512cc2-9edb-4a62-b18d-93f4445528a6 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:32:58.649 257256 INFO cinder.volume.manager [req-19512cc2-9edb-4a62-b18d-93f4445528a6 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Copy volume to image completed successfully. >2018-08-24 03:33:01.767 257256 INFO cinder.volume.manager [req-3fe53e9e-f5de-4277-939b-b863b9a14a07 58684bafa5ef4654ae284766b5dccd2c 10e6d7a2b33d43f4b5395388b83fb463 - default default] Deleted volume successfully. >2018-08-24 03:33:08.247 257256 INFO cinder.volume.flows.manager.create_volume [req-cacf32a8-c2cd-48a9-9291-3eb0cad38dda 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Volume ea6fb28e-0f46-4b08-b53d-41df5a943320: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ea6fb28e-0f46-4b08-b53d-41df5a943320'} >2018-08-24 03:33:08.772 257256 INFO cinder.volume.flows.manager.create_volume [req-cacf32a8-c2cd-48a9-9291-3eb0cad38dda 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Volume volume-ea6fb28e-0f46-4b08-b53d-41df5a943320 (ea6fb28e-0f46-4b08-b53d-41df5a943320): created successfully >2018-08-24 03:33:08.776 257256 INFO cinder.volume.manager [req-cacf32a8-c2cd-48a9-9291-3eb0cad38dda 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Created volume successfully. >2018-08-24 03:33:25.448 257256 INFO cinder.volume.manager [req-84f7f108-97bb-45b9-9aac-a8c3fdd4c835 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Initialize volume connection completed successfully. >2018-08-24 03:33:29.944 257256 INFO cinder.volume.manager [req-e22aeb43-0688-4587-802e-64e374437d12 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Attaching volume ea6fb28e-0f46-4b08-b53d-41df5a943320 to instance 201a649c-e261-4d2d-b247-4497ce90b23c at mountpoint /dev/vdb on host None. >2018-08-24 03:33:30.018 257256 INFO cinder.volume.manager [req-e22aeb43-0688-4587-802e-64e374437d12 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Attach volume completed successfully. >2018-08-24 03:33:32.266 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e354d500-4259-40c4-a185-a4e632adad07 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Unable to get remote copy information for volume volume-ea6fb28e-0f46-4b08-b53d-41df5a943320 >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova [req-e354d500-4259-40c4-a185-a4e632adad07 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Failed to notify nova on events: [{'tag': 'ea6fb28e-0f46-4b08-b53d-41df5a943320', 'name': 'volume-extended', 'server_uuid': '201a649c-e261-4d2d-b247-4497ce90b23c'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-91d08a7d-f5f3-4a6c-9343-48787c8d5d3a) >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova return_raw=True) >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-91d08a7d-f5f3-4a6c-9343-48787c8d5d3a) >2018-08-24 03:33:33.188 257256 ERROR cinder.compute.nova >2018-08-24 03:33:33.212 257256 INFO cinder.volume.manager [req-e354d500-4259-40c4-a185-a4e632adad07 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Extend volume completed successfully. >2018-08-24 03:38:45.188 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-bc763ea5-1043-421e-a0a2-cf05a9f6cbee 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:38:45.467 257256 INFO cinder.volume.manager [req-bc763ea5-1043-421e-a0a2-cf05a9f6cbee 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Terminate volume connection completed successfully. >2018-08-24 03:38:45.805 257256 INFO cinder.volume.manager [req-a715a956-1a2e-415f-9fef-a16677f3adf9 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Detaching volume ea6fb28e-0f46-4b08-b53d-41df5a943320 from instance 201a649c-e261-4d2d-b247-4497ce90b23c. >2018-08-24 03:38:45.891 257256 INFO cinder.volume.manager [req-a715a956-1a2e-415f-9fef-a16677f3adf9 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Detach volume completed successfully. >2018-08-24 03:38:49.617 257256 INFO cinder.volume.manager [req-c404e212-6d08-4f5c-a63a-3dc36f7529eb 3838ca95250245dba8fed27a1f333267 85925eef9c974d7fb57c8193980dcadd - default default] Deleted volume successfully. >2018-08-24 03:38:54.833 257256 INFO cinder.volume.flows.manager.create_volume [req-8ba85f95-181f-44a5-a8e7-689185b7d5e1 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] Volume 5b69bf0c-ae99-40c4-aae3-6f63b6139303: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5b69bf0c-ae99-40c4-aae3-6f63b6139303'} >2018-08-24 03:38:55.361 257256 INFO cinder.volume.flows.manager.create_volume [req-8ba85f95-181f-44a5-a8e7-689185b7d5e1 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] Volume volume-5b69bf0c-ae99-40c4-aae3-6f63b6139303 (5b69bf0c-ae99-40c4-aae3-6f63b6139303): created successfully >2018-08-24 03:38:55.364 257256 INFO cinder.volume.manager [req-8ba85f95-181f-44a5-a8e7-689185b7d5e1 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] Created volume successfully. >2018-08-24 03:38:56.965 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-185f890f-b087-4aa6-a66e-fd4fa9570e55 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] Unable to get remote copy information for volume volume-5b69bf0c-ae99-40c4-aae3-6f63b6139303 >2018-08-24 03:38:57.244 257256 INFO cinder.volume.manager [req-185f890f-b087-4aa6-a66e-fd4fa9570e55 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] Extend volume completed successfully. >2018-08-24 03:38:59.584 257256 INFO cinder.volume.manager [req-d6309a5f-664d-4b38-9a6b-5aac727c1970 99b5b046f376410680a2ce8e5cb3535c 9eb8208f59fb4261a97c099cb1b5fce5 - default default] Deleted volume successfully. >2018-08-24 03:39:03.991 257256 INFO cinder.volume.flows.manager.create_volume [req-7d413806-6d06-4a26-b98e-d97b770483aa eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume a9f336d6-ba03-47f0-a114-a4da8cdd3d02: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a9f336d6-ba03-47f0-a114-a4da8cdd3d02'} >2018-08-24 03:39:04.514 257256 INFO cinder.volume.flows.manager.create_volume [req-7d413806-6d06-4a26-b98e-d97b770483aa eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume volume-a9f336d6-ba03-47f0-a114-a4da8cdd3d02 (a9f336d6-ba03-47f0-a114-a4da8cdd3d02): created successfully >2018-08-24 03:39:04.518 257256 INFO cinder.volume.manager [req-7d413806-6d06-4a26-b98e-d97b770483aa eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Created volume successfully. >2018-08-24 03:39:06.984 257256 INFO cinder.volume.flows.manager.create_volume [req-a5b97967-3a48-45d2-b63b-eabc8e0899bd eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume 1b55e7e9-ae3d-4482-b7eb-cb3207a9de7e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1b55e7e9-ae3d-4482-b7eb-cb3207a9de7e'} >2018-08-24 03:39:07.509 257256 INFO cinder.volume.flows.manager.create_volume [req-a5b97967-3a48-45d2-b63b-eabc8e0899bd eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume volume-1b55e7e9-ae3d-4482-b7eb-cb3207a9de7e (1b55e7e9-ae3d-4482-b7eb-cb3207a9de7e): created successfully >2018-08-24 03:39:07.512 257256 INFO cinder.volume.manager [req-a5b97967-3a48-45d2-b63b-eabc8e0899bd eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Created volume successfully. >2018-08-24 03:39:09.477 257256 INFO cinder.volume.manager [req-0465be08-0cc2-46e5-a8b4-5f2752a1b608 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Deleted volume successfully. >2018-08-24 03:39:11.351 257256 INFO cinder.volume.manager [req-4e68e9a9-94be-4e8d-b160-c9353432ee8e eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Deleted volume successfully. >2018-08-24 03:39:11.686 257256 INFO cinder.volume.flows.manager.create_volume [req-68d76243-f6ae-4802-9b3b-616607df1c8e eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume 1db30a6f-4823-4627-a5e1-44ad954b78b1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1db30a6f-4823-4627-a5e1-44ad954b78b1'} >2018-08-24 03:39:12.212 257256 INFO cinder.volume.flows.manager.create_volume [req-68d76243-f6ae-4802-9b3b-616607df1c8e eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume volume-1db30a6f-4823-4627-a5e1-44ad954b78b1 (1db30a6f-4823-4627-a5e1-44ad954b78b1): created successfully >2018-08-24 03:39:12.216 257256 INFO cinder.volume.manager [req-68d76243-f6ae-4802-9b3b-616607df1c8e eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Created volume successfully. >2018-08-24 03:39:13.107 257256 INFO cinder.volume.flows.manager.create_volume [req-f389fbfc-1ffd-4908-b58b-0387eb023845 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume 64f6073e-a454-4333-a055-604c990892a7: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-64f6073e-a454-4333-a055-604c990892a7', 'source_volid': '1db30a6f-4823-4627-a5e1-44ad954b78b1', 'source_volstatus': u'available'} >2018-08-24 03:39:14.599 257256 INFO cinder.volume.flows.manager.create_volume [req-f389fbfc-1ffd-4908-b58b-0387eb023845 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume volume-64f6073e-a454-4333-a055-604c990892a7 (64f6073e-a454-4333-a055-604c990892a7): created successfully >2018-08-24 03:39:14.603 257256 INFO cinder.volume.manager [req-f389fbfc-1ffd-4908-b58b-0387eb023845 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Created volume successfully. >2018-08-24 03:39:15.930 257256 INFO cinder.volume.flows.manager.create_volume [req-d7b43313-1c68-4894-95ae-939113526cca eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume ea0e8532-5bc8-4806-96e0-824944c51710: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ea0e8532-5bc8-4806-96e0-824944c51710'} >2018-08-24 03:39:16.462 257256 INFO cinder.volume.flows.manager.create_volume [req-d7b43313-1c68-4894-95ae-939113526cca eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume volume-ea0e8532-5bc8-4806-96e0-824944c51710 (ea0e8532-5bc8-4806-96e0-824944c51710): created successfully >2018-08-24 03:39:16.466 257256 INFO cinder.volume.manager [req-d7b43313-1c68-4894-95ae-939113526cca eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Created volume successfully. >2018-08-24 03:39:18.355 257256 INFO cinder.volume.manager [req-d37e6d22-285e-479e-aad7-7e9b1adf5b1d eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Deleted volume successfully. >2018-08-24 03:39:22.938 257256 INFO cinder.volume.manager [req-adfa6a2c-3417-4197-8b11-5cf1167399b3 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Deleted volume successfully. >2018-08-24 03:39:23.776 257256 INFO cinder.volume.flows.manager.create_volume [req-3d196be1-b812-476e-8a68-5e3b8878eacc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume d9781bc0-1c87-479e-8f37-4c39cc29270c: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-d9781bc0-1c87-479e-8f37-4c39cc29270c', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab541fd0d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-24 03:39:24.577 257256 INFO cinder.image.image_utils [req-3d196be1-b812-476e-8a68-5e3b8878eacc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-24 03:39:36.938 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 03:39:41.873 257256 INFO cinder.image.image_utils [req-3d196be1-b812-476e-8a68-5e3b8878eacc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-24 03:39:43.714 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-3d196be1-b812-476e-8a68-5e3b8878eacc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:39:44.065 257256 INFO cinder.volume.flows.manager.create_volume [req-3d196be1-b812-476e-8a68-5e3b8878eacc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume volume-d9781bc0-1c87-479e-8f37-4c39cc29270c (d9781bc0-1c87-479e-8f37-4c39cc29270c): created successfully >2018-08-24 03:39:44.069 257256 INFO cinder.volume.manager [req-3d196be1-b812-476e-8a68-5e3b8878eacc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Created volume successfully. >2018-08-24 03:39:44.860 257256 INFO cinder.volume.flows.manager.create_volume [req-fa5de27e-c527-46d1-a9b5-36e2d4badd87 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume b0d8d350-b05e-463d-8a0d-dc8579e31684: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b0d8d350-b05e-463d-8a0d-dc8579e31684'} >2018-08-24 03:39:45.396 257256 INFO cinder.volume.flows.manager.create_volume [req-fa5de27e-c527-46d1-a9b5-36e2d4badd87 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Volume volume-b0d8d350-b05e-463d-8a0d-dc8579e31684 (b0d8d350-b05e-463d-8a0d-dc8579e31684): created successfully >2018-08-24 03:39:45.399 257256 INFO cinder.volume.manager [req-fa5de27e-c527-46d1-a9b5-36e2d4badd87 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Created volume successfully. >2018-08-24 03:39:47.212 257256 INFO cinder.volume.manager [req-b91c46d4-2386-4988-8338-e0cc66b78ffc eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Deleted volume successfully. >2018-08-24 03:39:48.503 257256 INFO cinder.volume.manager [req-8df00e56-d2e8-46b2-afc8-32caa4bb1dc0 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Deleted volume successfully. >2018-08-24 03:39:49.683 257256 INFO cinder.volume.manager [req-ae585b72-f743-4371-87e7-1bd0523102f2 eee084769eb04fcb841ae13b5738b81d e1e6fbe8498d4ab3a33259ced1b79048 - default default] Deleted volume successfully. >2018-08-24 03:39:58.066 257256 INFO cinder.volume.flows.manager.create_volume [req-af0189b3-02e3-4593-bdd7-132211f329fd 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Volume 56bc8998-53cc-4469-ae99-24a2cecf78d9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-56bc8998-53cc-4469-ae99-24a2cecf78d9'} >2018-08-24 03:39:58.589 257256 INFO cinder.volume.flows.manager.create_volume [req-af0189b3-02e3-4593-bdd7-132211f329fd 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Volume volume-56bc8998-53cc-4469-ae99-24a2cecf78d9 (56bc8998-53cc-4469-ae99-24a2cecf78d9): created successfully >2018-08-24 03:39:58.592 257256 INFO cinder.volume.manager [req-af0189b3-02e3-4593-bdd7-132211f329fd 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Created volume successfully. >2018-08-24 03:40:00.118 257256 INFO cinder.volume.flows.manager.create_volume [req-1f28a676-96ab-4c4c-bdec-eb0263386688 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Volume 73d2e17e-9d1a-44af-a024-bfa8b03ab96a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-73d2e17e-9d1a-44af-a024-bfa8b03ab96a'} >2018-08-24 03:40:00.641 257256 INFO cinder.volume.flows.manager.create_volume [req-1f28a676-96ab-4c4c-bdec-eb0263386688 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Volume volume-73d2e17e-9d1a-44af-a024-bfa8b03ab96a (73d2e17e-9d1a-44af-a024-bfa8b03ab96a): created successfully >2018-08-24 03:40:00.645 257256 INFO cinder.volume.manager [req-1f28a676-96ab-4c4c-bdec-eb0263386688 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Created volume successfully. >2018-08-24 03:40:02.087 257256 INFO cinder.volume.flows.manager.create_volume [req-c4f7c4a6-801a-45e3-a250-1b083787432c 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Volume ba38ecd8-68d6-40fd-a6de-98f0ff914db2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ba38ecd8-68d6-40fd-a6de-98f0ff914db2'} >2018-08-24 03:40:02.618 257256 INFO cinder.volume.flows.manager.create_volume [req-c4f7c4a6-801a-45e3-a250-1b083787432c 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Volume volume-ba38ecd8-68d6-40fd-a6de-98f0ff914db2 (ba38ecd8-68d6-40fd-a6de-98f0ff914db2): created successfully >2018-08-24 03:40:02.622 257256 INFO cinder.volume.manager [req-c4f7c4a6-801a-45e3-a250-1b083787432c 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Created volume successfully. >2018-08-24 03:40:09.009 257256 INFO cinder.volume.manager [req-cf5bff0b-6a33-4787-a925-c8a49e91acbd 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Deleted volume successfully. >2018-08-24 03:40:09.031 257256 INFO cinder.volume.manager [req-828e7aa4-7b4e-4f48-8d15-51fb3c722c08 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Deleted volume successfully. >2018-08-24 03:40:09.264 257256 INFO cinder.volume.manager [req-9ef0437a-2cc8-453d-a644-a8938be9e9a7 1091616d0d6043a7881c667fb30c40ba b8920fe6be3d40bb83fa9526359077a7 - default default] Deleted volume successfully. >2018-08-24 03:40:13.483 257256 INFO cinder.volume.flows.manager.create_volume [req-29dfc441-53ad-431e-ab77-f2700fde1c74 e8e81662cd6d46e0a700ac637b790df0 18d326aa78de42a295343b8b6177101c - default default] Volume 7a77da64-11f5-449c-a0f0-c8fc6343bb6b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7a77da64-11f5-449c-a0f0-c8fc6343bb6b'} >2018-08-24 03:40:14.005 257256 INFO cinder.volume.flows.manager.create_volume [req-29dfc441-53ad-431e-ab77-f2700fde1c74 e8e81662cd6d46e0a700ac637b790df0 18d326aa78de42a295343b8b6177101c - default default] Volume volume-7a77da64-11f5-449c-a0f0-c8fc6343bb6b (7a77da64-11f5-449c-a0f0-c8fc6343bb6b): created successfully >2018-08-24 03:40:14.009 257256 INFO cinder.volume.manager [req-29dfc441-53ad-431e-ab77-f2700fde1c74 e8e81662cd6d46e0a700ac637b790df0 18d326aa78de42a295343b8b6177101c - default default] Created volume successfully. >2018-08-24 03:40:34.622 257256 INFO cinder.volume.manager [req-5cac2855-8737-4593-b2a0-3d437cda0166 e8e81662cd6d46e0a700ac637b790df0 18d326aa78de42a295343b8b6177101c - default default] Deleted volume successfully. >2018-08-24 03:40:39.766 257256 INFO cinder.volume.flows.manager.create_volume [req-4c325d9b-05f7-4848-963f-8882fb5e5f0f f0345b5bc3214de7ae5865775759a054 a7124cc6b458421a8f33d4a3212174c5 - default default] Volume 334296cf-cd90-4680-8922-cbe2ec762ee3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-334296cf-cd90-4680-8922-cbe2ec762ee3'} >2018-08-24 03:40:40.634 257256 INFO cinder.volume.flows.manager.create_volume [req-4c325d9b-05f7-4848-963f-8882fb5e5f0f f0345b5bc3214de7ae5865775759a054 a7124cc6b458421a8f33d4a3212174c5 - default default] Volume volume-334296cf-cd90-4680-8922-cbe2ec762ee3 (334296cf-cd90-4680-8922-cbe2ec762ee3): created successfully >2018-08-24 03:40:40.638 257256 INFO cinder.volume.manager [req-4c325d9b-05f7-4848-963f-8882fb5e5f0f f0345b5bc3214de7ae5865775759a054 a7124cc6b458421a8f33d4a3212174c5 - default default] Created volume successfully. >2018-08-24 03:40:43.749 257256 INFO cinder.volume.manager [req-5260f236-63ab-4f5f-b37e-fdfc92478428 f0345b5bc3214de7ae5865775759a054 a7124cc6b458421a8f33d4a3212174c5 - default default] Deleted volume successfully. >2018-08-24 03:41:31.346 257256 INFO cinder.volume.flows.manager.create_volume [req-e9c7f1aa-11ff-45a8-84a2-e89b6f308a66 dafd7e968dbd449aa7b0f083f7f4a759 07a5aa1f42904e8fb04d13f4993ea959 - default default] Volume a74d28c2-ebce-4550-8554-789f53fa6064: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a74d28c2-ebce-4550-8554-789f53fa6064'} >2018-08-24 03:41:31.880 257256 INFO cinder.volume.flows.manager.create_volume [req-e9c7f1aa-11ff-45a8-84a2-e89b6f308a66 dafd7e968dbd449aa7b0f083f7f4a759 07a5aa1f42904e8fb04d13f4993ea959 - default default] Volume volume-a74d28c2-ebce-4550-8554-789f53fa6064 (a74d28c2-ebce-4550-8554-789f53fa6064): created successfully >2018-08-24 03:41:31.885 257256 INFO cinder.volume.manager [req-e9c7f1aa-11ff-45a8-84a2-e89b6f308a66 dafd7e968dbd449aa7b0f083f7f4a759 07a5aa1f42904e8fb04d13f4993ea959 - default default] Created volume successfully. >2018-08-24 03:41:34.808 257256 INFO cinder.volume.manager [req-b13d36f9-50e3-4403-9b55-5dd4bd125c42 dafd7e968dbd449aa7b0f083f7f4a759 07a5aa1f42904e8fb04d13f4993ea959 - default default] Deleted volume successfully. >2018-08-24 03:41:40.273 257256 INFO cinder.volume.flows.manager.create_volume [req-a609e88a-5f99-4238-a026-a5c554920b49 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Volume 54ed9eca-ab2b-4270-a587-c57d669864c4: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-54ed9eca-ab2b-4270-a587-c57d669864c4', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54747c90>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-24 03:41:41.324 257256 INFO cinder.image.image_utils [req-a609e88a-5f99-4238-a026-a5c554920b49 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Image download 12.00 MB at 11.43 MB/s >2018-08-24 03:41:50.122 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 03:41:55.509 257256 INFO cinder.image.image_utils [req-a609e88a-5f99-4238-a026-a5c554920b49 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-24 03:41:57.386 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-a609e88a-5f99-4238-a026-a5c554920b49 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:41:57.736 257256 INFO cinder.volume.flows.manager.create_volume [req-a609e88a-5f99-4238-a026-a5c554920b49 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Volume volume-54ed9eca-ab2b-4270-a587-c57d669864c4 (54ed9eca-ab2b-4270-a587-c57d669864c4): created successfully >2018-08-24 03:41:57.740 257256 INFO cinder.volume.manager [req-a609e88a-5f99-4238-a026-a5c554920b49 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Created volume successfully. >2018-08-24 03:41:58.759 257256 INFO cinder.volume.flows.manager.create_volume [req-3050beec-421f-439c-973d-89a6afa340a8 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Volume 70fd2d1e-3ad0-41f5-b9d9-56b559f2ef7d: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-70fd2d1e-3ad0-41f5-b9d9-56b559f2ef7d', 'source_volid': '54ed9eca-ab2b-4270-a587-c57d669864c4', 'source_volstatus': u'available'} >2018-08-24 03:42:00.400 257256 INFO cinder.volume.flows.manager.create_volume [req-3050beec-421f-439c-973d-89a6afa340a8 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Volume volume-70fd2d1e-3ad0-41f5-b9d9-56b559f2ef7d (70fd2d1e-3ad0-41f5-b9d9-56b559f2ef7d): created successfully >2018-08-24 03:42:00.404 257256 INFO cinder.volume.manager [req-3050beec-421f-439c-973d-89a6afa340a8 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Created volume successfully. >2018-08-24 03:42:04.157 257256 INFO cinder.volume.manager [req-b1b8de78-c85b-4c2b-8928-feaeca57280e c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Deleted volume successfully. >2018-08-24 03:42:05.632 257256 INFO cinder.volume.manager [req-161b60b2-8b84-4739-a9d4-f05341bf5cd6 c67b6e2c710c46b9b1200d88188a598a ecfda55236f34868a5da996e0135104e - default default] Deleted volume successfully. >2018-08-24 03:42:10.413 257256 INFO cinder.volume.flows.manager.create_volume [req-e18bb8e4-9467-4ea4-a9dc-d613aa9d53fc d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Volume 3c8219fb-7662-4797-9324-c675358b2137: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3c8219fb-7662-4797-9324-c675358b2137'} >2018-08-24 03:42:10.952 257256 INFO cinder.volume.flows.manager.create_volume [req-e18bb8e4-9467-4ea4-a9dc-d613aa9d53fc d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Volume volume-3c8219fb-7662-4797-9324-c675358b2137 (3c8219fb-7662-4797-9324-c675358b2137): created successfully >2018-08-24 03:42:10.957 257256 INFO cinder.volume.manager [req-e18bb8e4-9467-4ea4-a9dc-d613aa9d53fc d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Created volume successfully. >2018-08-24 03:42:12.716 257256 INFO cinder.volume.flows.manager.create_volume [req-3e8217ce-39d5-4c26-8a6f-f0230c18db45 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Volume e306833e-319a-4275-ba5a-7912d078c7da: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e306833e-319a-4275-ba5a-7912d078c7da'} >2018-08-24 03:42:13.280 257256 INFO cinder.volume.flows.manager.create_volume [req-3e8217ce-39d5-4c26-8a6f-f0230c18db45 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Volume volume-e306833e-319a-4275-ba5a-7912d078c7da (e306833e-319a-4275-ba5a-7912d078c7da): created successfully >2018-08-24 03:42:13.284 257256 INFO cinder.volume.manager [req-3e8217ce-39d5-4c26-8a6f-f0230c18db45 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Created volume successfully. >2018-08-24 03:42:14.601 257031 INFO cinder.volume.flows.manager.create_volume [req-a74fdb73-b174-4ff1-9150-953776dd6019 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Volume 50650ea8-9072-44e8-8da4-b30f0b7e080d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-50650ea8-9072-44e8-8da4-b30f0b7e080d'} >2018-08-24 03:42:15.140 257031 INFO cinder.volume.flows.manager.create_volume [req-a74fdb73-b174-4ff1-9150-953776dd6019 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Volume volume-50650ea8-9072-44e8-8da4-b30f0b7e080d (50650ea8-9072-44e8-8da4-b30f0b7e080d): created successfully >2018-08-24 03:42:15.144 257031 INFO cinder.volume.manager [req-a74fdb73-b174-4ff1-9150-953776dd6019 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Created volume successfully. >2018-08-24 03:42:18.782 257256 INFO cinder.volume.manager [req-d3cd17d7-dae9-4e32-8b62-2ab0aa747f6b d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Deleted volume successfully. >2018-08-24 03:42:18.801 257256 INFO cinder.volume.manager [req-10fcd2c6-73ac-4dd4-b1bb-897d3832a3e4 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Deleted volume successfully. >2018-08-24 03:42:19.030 257031 INFO cinder.volume.manager [req-820f3c32-1421-4eae-bad9-4d35bef17dd0 d2e2fb0abe264dbcba10777334cb7dd8 a0d6efc1fdcd4a889120abbf5987a4fc - default default] Deleted volume successfully. >2018-08-24 03:42:39.772 257031 INFO cinder.volume.flows.manager.create_volume [req-6eba422a-5d54-4d58-bce9-7bbff40522da 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Volume 20295c59-8045-45c4-af1d-bdeea1dc0d51: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-20295c59-8045-45c4-af1d-bdeea1dc0d51'} >2018-08-24 03:42:40.313 257031 INFO cinder.volume.flows.manager.create_volume [req-6eba422a-5d54-4d58-bce9-7bbff40522da 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Volume volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 (20295c59-8045-45c4-af1d-bdeea1dc0d51): created successfully >2018-08-24 03:42:40.317 257031 INFO cinder.volume.manager [req-6eba422a-5d54-4d58-bce9-7bbff40522da 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Created volume successfully. >2018-08-24 03:42:42.194 257256 INFO cinder.volume.flows.manager.create_volume [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Volume b7f807e8-6c30-4331-8ee9-6720bae5bc49: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49'} >2018-08-24 03:42:42.731 257256 INFO cinder.volume.flows.manager.create_volume [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Volume volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49 (b7f807e8-6c30-4331-8ee9-6720bae5bc49): created successfully >2018-08-24 03:42:42.735 257256 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Created volume successfully. >2018-08-24 03:42:49.921 257256 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Initialize volume connection completed successfully. >2018-08-24 03:42:50.312 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 03:42:53.876 257031 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Initialize volume connection completed successfully. >2018-08-24 03:42:54.262 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 03:43:00.918 257031 INFO cinder.volume.utils [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Volume copy 1024.00 MB at 278.66 MB/s >2018-08-24 03:43:02.818 257256 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Terminate volume connection completed successfully. >2018-08-24 03:43:02.859 257256 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Remove volume export completed successfully. >2018-08-24 03:43:04.716 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:43:04.992 257031 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Terminate volume connection completed successfully. >2018-08-24 03:43:05.034 257031 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Remove volume export completed successfully. >2018-08-24 03:43:05.291 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 03:43:08.533 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Error running SSH command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 03:43:08.535 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 03:43:08.535 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Error running SSH command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 03:43:08.536 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-20295c59-8045-45c4-af1d-bdeea1dc0d51', '"volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-20295c59-8045-45c4-af1d-bdeea1dc0d51 "volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 03:43:08.537 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Unable to rename the logical volume for volume: 20295c59-8045-45c4-af1d-bdeea1dc0d51: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-20295c59-8045-45c4-af1d-bdeea1dc0d51', '"volume-b7f807e8-6c30-4331-8ee9-6720bae5bc49"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-24 03:43:08.711 257031 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Complete-Migrate volume completed successfully. >2018-08-24 03:43:08.712 257031 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Migrate volume completed successfully. >2018-08-24 03:43:09.240 257031 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Retype volume completed successfully. >2018-08-24 03:43:09.985 257031 INFO cinder.volume.manager [req-e25f8bf8-37ba-4ab4-b060-8af9300da363 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Deleted volume successfully. >2018-08-24 03:43:11.794 257256 WARNING cinder.quota [req-ba20a235-2ffa-40df-93bb-b8d61591c098 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1046939679 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1046939679, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 03:43:11.795 257256 WARNING cinder.quota [req-ba20a235-2ffa-40df-93bb-b8d61591c098 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1046939679 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1046939679, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 03:43:12.050 257256 INFO cinder.volume.manager [req-ba20a235-2ffa-40df-93bb-b8d61591c098 7baf6baf56af4622a5098b1619d40543 8d17afd1302b4cb18bd7f547cfc5c292 - default default] Deleted volume successfully. >2018-08-24 03:43:33.160 257256 INFO cinder.volume.flows.manager.create_volume [req-0d54915d-3d55-49e9-a84d-9b6a61726bd6 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Volume 89a0f221-dc81-49bb-92cf-e045d15e109b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-89a0f221-dc81-49bb-92cf-e045d15e109b'} >2018-08-24 03:43:33.799 257256 INFO cinder.volume.flows.manager.create_volume [req-0d54915d-3d55-49e9-a84d-9b6a61726bd6 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Volume volume-89a0f221-dc81-49bb-92cf-e045d15e109b (89a0f221-dc81-49bb-92cf-e045d15e109b): created successfully >2018-08-24 03:43:33.803 257256 INFO cinder.volume.manager [req-0d54915d-3d55-49e9-a84d-9b6a61726bd6 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Created volume successfully. >2018-08-24 03:43:36.463 257256 INFO cinder.volume.manager [req-b9017f88-6906-48b1-ad65-cfd205c69956 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Create snapshot completed successfully >2018-08-24 03:43:39.032 257256 INFO cinder.volume.manager [req-ae0cb58b-012f-4726-a8db-09a3bb1031ac 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Create snapshot completed successfully >2018-08-24 03:43:41.477 257256 INFO cinder.volume.manager [req-90f281fc-7743-4860-b75f-9e302894f717 a0df296f539745b9aa4919951180cee3 9567c3f3af3a4391b5664dd21bf8aa39 - default default] Delete snapshot completed successfully. >2018-08-24 03:43:44.199 257256 INFO cinder.volume.manager [req-36c67064-df54-4b02-be1a-20e3d0ceef92 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Create snapshot completed successfully >2018-08-24 03:43:47.519 257256 INFO cinder.volume.manager [req-cfb92a9f-0c78-49c0-adad-b90a0787ea10 a0df296f539745b9aa4919951180cee3 9567c3f3af3a4391b5664dd21bf8aa39 - default default] Delete snapshot completed successfully. >2018-08-24 03:43:50.124 257256 INFO cinder.volume.manager [req-727291d2-55f5-4dc5-a44b-1dcd9da094d2 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Create snapshot completed successfully >2018-08-24 03:43:53.421 257256 INFO cinder.volume.manager [req-64a6e791-3b32-4b05-940f-7ebd36c7876c a0df296f539745b9aa4919951180cee3 9567c3f3af3a4391b5664dd21bf8aa39 - default default] Delete snapshot completed successfully. >2018-08-24 03:43:55.385 257256 INFO cinder.volume.manager [req-f1cc98c6-9493-4092-ae8f-5f76007e4467 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Create snapshot completed successfully >2018-08-24 03:43:57.939 257256 INFO cinder.volume.manager [req-dc380790-39b7-4dcf-8794-d6bedb79e411 a0df296f539745b9aa4919951180cee3 9567c3f3af3a4391b5664dd21bf8aa39 - default default] Delete snapshot completed successfully. >2018-08-24 03:44:00.636 257256 INFO cinder.volume.manager [req-ae7d1ed5-8e9b-40f5-9066-e894b77a4968 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Delete snapshot completed successfully. >2018-08-24 03:44:02.652 257256 INFO cinder.volume.manager [req-13596863-5a83-43dd-9fec-ebd427b29206 61bfb6e20cdc4aac89d434c3e01e381e d7bd0122bccd44ee8d49053ab44acb82 - default default] Deleted volume successfully. >2018-08-24 03:44:07.831 257256 INFO cinder.volume.flows.manager.create_volume [req-bd9dc69a-2c9c-44b4-a98e-684c46eaeefb 339512fdfbd84279870f7a186ed5fef2 f5756552fd3240278a9e81d1b0a9f7f0 - default default] Volume 8ce2aac2-22fe-44c2-b799-99189c149de4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8ce2aac2-22fe-44c2-b799-99189c149de4'} >2018-08-24 03:44:08.367 257256 INFO cinder.volume.flows.manager.create_volume [req-bd9dc69a-2c9c-44b4-a98e-684c46eaeefb 339512fdfbd84279870f7a186ed5fef2 f5756552fd3240278a9e81d1b0a9f7f0 - default default] Volume volume-8ce2aac2-22fe-44c2-b799-99189c149de4 (8ce2aac2-22fe-44c2-b799-99189c149de4): created successfully >2018-08-24 03:44:08.371 257256 INFO cinder.volume.manager [req-bd9dc69a-2c9c-44b4-a98e-684c46eaeefb 339512fdfbd84279870f7a186ed5fef2 f5756552fd3240278a9e81d1b0a9f7f0 - default default] Created volume successfully. >2018-08-24 03:44:11.002 257256 INFO cinder.volume.manager [req-a7a56362-d7f5-4b75-bad4-174e30f4db06 339512fdfbd84279870f7a186ed5fef2 f5756552fd3240278a9e81d1b0a9f7f0 - default default] Create snapshot completed successfully >2018-08-24 03:44:16.073 257256 INFO cinder.volume.manager [req-c77091eb-b3a8-4f84-9dd7-83eb1662f86c 339512fdfbd84279870f7a186ed5fef2 f5756552fd3240278a9e81d1b0a9f7f0 - default default] Delete snapshot completed successfully. >2018-08-24 03:44:17.560 257256 INFO cinder.volume.manager [req-468565b3-4c2a-4629-b685-23bbbd14e375 339512fdfbd84279870f7a186ed5fef2 f5756552fd3240278a9e81d1b0a9f7f0 - default default] Deleted volume successfully. >2018-08-24 03:44:21.869 257256 INFO cinder.volume.flows.manager.create_volume [req-3762c952-22b6-4828-bfc1-1e80fc3ee166 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Volume 5024b0e2-dfd3-4f95-bd79-0c07b6599ed6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5024b0e2-dfd3-4f95-bd79-0c07b6599ed6'} >2018-08-24 03:44:22.407 257256 INFO cinder.volume.flows.manager.create_volume [req-3762c952-22b6-4828-bfc1-1e80fc3ee166 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Volume volume-5024b0e2-dfd3-4f95-bd79-0c07b6599ed6 (5024b0e2-dfd3-4f95-bd79-0c07b6599ed6): created successfully >2018-08-24 03:44:22.411 257256 INFO cinder.volume.manager [req-3762c952-22b6-4828-bfc1-1e80fc3ee166 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Created volume successfully. >2018-08-24 03:44:28.784 257256 INFO cinder.volume.manager [req-ed260643-18da-4cfd-bf77-ab20c346e0c6 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:44:43.753 257256 INFO cinder.volume.manager [req-77b3c804-2845-45fd-b896-5db3b917ed80 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Initialize volume connection completed successfully. >2018-08-24 03:45:07.156 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-77b3c804-2845-45fd-b896-5db3b917ed80 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:45:07.563 257256 INFO cinder.volume.manager [req-77b3c804-2845-45fd-b896-5db3b917ed80 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Terminate volume connection completed successfully. >2018-08-24 03:45:07.605 257256 INFO cinder.volume.manager [req-77b3c804-2845-45fd-b896-5db3b917ed80 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Remove volume export completed successfully. >2018-08-24 03:45:12.598 257256 INFO cinder.volume.manager [req-77b3c804-2845-45fd-b896-5db3b917ed80 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Deleted volume successfully. >2018-08-24 03:45:25.945 257256 INFO cinder.volume.manager [req-a005ceb9-153d-4550-9802-65eb2f657c29 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Initialize volume connection completed successfully. >2018-08-24 03:45:30.496 257256 INFO cinder.volume.manager [req-eb38b451-3687-443c-b6e5-30b1641faefa 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Attaching volume 5024b0e2-dfd3-4f95-bd79-0c07b6599ed6 to instance 102d5441-494f-44c7-ad65-c0e8304b5b0e at mountpoint /dev/vdb on host None. >2018-08-24 03:45:30.571 257256 INFO cinder.volume.manager [req-eb38b451-3687-443c-b6e5-30b1641faefa 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Attach volume completed successfully. >2018-08-24 03:45:33.700 257256 INFO cinder.volume.manager [req-70e09ebf-f23a-4fc5-92eb-d3602a663d58 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:45:35.709 257256 INFO cinder.volume.manager [req-7334ef57-c4f5-46fa-9039-d60783e6e383 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:45:38.325 257256 INFO cinder.volume.manager [req-ba275d84-3000-416d-8b25-938fd98c5bcc 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:45:40.430 257256 INFO cinder.volume.manager [req-33ebfda6-168d-403a-9b2b-d89d086f01ed 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:45:42.637 257256 INFO cinder.volume.manager [req-0096f23c-ae43-4643-ab9f-b510fd228613 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:45:44.889 257256 INFO cinder.volume.manager [req-73a904fd-4a30-48c7-bfaa-cc00815c9890 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:45:55.920 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-1055f66e-73ab-4125-90dc-0cdd18645abe 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:45:56.213 257256 INFO cinder.volume.manager [req-1055f66e-73ab-4125-90dc-0cdd18645abe 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Terminate volume connection completed successfully. >2018-08-24 03:45:56.334 257256 INFO cinder.volume.manager [req-026dcf0b-2449-4bf5-a87e-65fb703a181e 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Detaching volume 5024b0e2-dfd3-4f95-bd79-0c07b6599ed6 from instance 102d5441-494f-44c7-ad65-c0e8304b5b0e. >2018-08-24 03:45:56.414 257256 INFO cinder.volume.manager [req-026dcf0b-2449-4bf5-a87e-65fb703a181e 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Detach volume completed successfully. >2018-08-24 03:46:00.498 257256 INFO cinder.volume.manager [req-36df04fa-45f9-45dc-b9a8-0b52a1410f76 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:46:03.168 257256 INFO cinder.volume.manager [req-5b5280ad-8f12-4fc4-abb8-9a74591b7990 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:46:05.729 257256 INFO cinder.volume.manager [req-6ee13d9c-92f0-47e7-aac5-042075fd82fd 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:46:24.776 257256 INFO cinder.volume.manager [req-2a6a525b-fb67-4911-84c2-d929adc0d530 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Initialize volume connection completed successfully. >2018-08-24 03:46:29.246 257256 INFO cinder.volume.manager [req-1a49e933-8493-4936-8990-e78d6e94cc6c 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Attaching volume 5024b0e2-dfd3-4f95-bd79-0c07b6599ed6 to instance a896946c-8c87-4de6-bf1b-11cc48cc8f5c at mountpoint /dev/vdb on host None. >2018-08-24 03:46:29.319 257256 INFO cinder.volume.manager [req-1a49e933-8493-4936-8990-e78d6e94cc6c 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Attach volume completed successfully. >2018-08-24 03:46:31.448 257256 INFO cinder.volume.manager [req-7c3767d8-78b2-4067-a0f9-6e61c9f7455f 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:46:33.471 257256 INFO cinder.volume.manager [req-8c0b1ea7-b4cc-44bf-806d-5dda40f40139 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:46:35.943 257256 INFO cinder.volume.manager [req-a80d7b58-959f-4008-90aa-da1f0f3ebc6f 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:46:38.375 257256 INFO cinder.volume.manager [req-fe4ab89d-3172-43ee-acd8-b8e79b15ce3a 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:46:42.973 257256 INFO cinder.volume.manager [req-0b216041-b1fc-47ce-baba-79963a148d3d 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:46:54.489 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-0edbc188-f754-4106-b6d2-372edfe1cf30 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:46:54.774 257256 INFO cinder.volume.manager [req-0edbc188-f754-4106-b6d2-372edfe1cf30 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Terminate volume connection completed successfully. >2018-08-24 03:46:54.921 257256 INFO cinder.volume.manager [req-e851485d-46ea-47ff-83b7-3e1b5b6e69f9 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Detaching volume 5024b0e2-dfd3-4f95-bd79-0c07b6599ed6 from instance a896946c-8c87-4de6-bf1b-11cc48cc8f5c. >2018-08-24 03:46:55.005 257256 INFO cinder.volume.manager [req-e851485d-46ea-47ff-83b7-3e1b5b6e69f9 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Detach volume completed successfully. >2018-08-24 03:46:57.562 257031 INFO cinder.volume.flows.manager.create_volume [req-416016ac-b225-4497-9267-406c8d17e503 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Volume 3fdb0d5d-d67f-44c9-a17e-f1cd873eedab: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3fdb0d5d-d67f-44c9-a17e-f1cd873eedab'} >2018-08-24 03:46:58.087 257031 INFO cinder.volume.flows.manager.create_volume [req-416016ac-b225-4497-9267-406c8d17e503 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Volume volume-3fdb0d5d-d67f-44c9-a17e-f1cd873eedab (3fdb0d5d-d67f-44c9-a17e-f1cd873eedab): created successfully >2018-08-24 03:46:58.091 257031 INFO cinder.volume.manager [req-416016ac-b225-4497-9267-406c8d17e503 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Created volume successfully. >2018-08-24 03:47:00.330 257031 INFO cinder.volume.manager [req-f356efb0-2dee-450f-8f0b-9e8454ea9f5b 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Create snapshot completed successfully >2018-08-24 03:47:01.237 257031 INFO cinder.volume.flows.manager.create_volume [req-bca06038-b5d1-4f4b-accf-8b328f7e5700 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Volume 94058fd6-3885-4427-8337-1670729aa64e: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': '02837b5d-2162-4c92-bbbc-68a50359ec73', 'volume_name': 'volume-94058fd6-3885-4427-8337-1670729aa64e'} >2018-08-24 03:47:30.242 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-bca06038-b5d1-4f4b-accf-8b328f7e5700 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Unable to get remote copy information for volume volume-94058fd6-3885-4427-8337-1670729aa64e >2018-08-24 03:47:30.554 257031 INFO cinder.volume.flows.manager.create_volume [req-bca06038-b5d1-4f4b-accf-8b328f7e5700 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Volume volume-94058fd6-3885-4427-8337-1670729aa64e (94058fd6-3885-4427-8337-1670729aa64e): created successfully >2018-08-24 03:47:30.559 257031 INFO cinder.volume.manager [req-bca06038-b5d1-4f4b-accf-8b328f7e5700 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Created volume successfully. >2018-08-24 03:47:33.160 257031 INFO cinder.volume.manager [req-51f05e56-7bae-4144-a8e3-284061d05818 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Deleted volume successfully. >2018-08-24 03:47:36.189 257256 INFO cinder.volume.manager [req-ff6d68a3-271d-4c08-b151-1e5294bc094b 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:47:36.490 257031 INFO cinder.volume.manager [req-21d6ab71-9cc8-4752-b583-bf64ae5b59e2 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Delete snapshot completed successfully. >2018-08-24 03:47:38.763 257031 WARNING cinder.db.sqlalchemy.api [req-8ed8e911-49bd-497a-ba19-f26a3b67c522 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-24 03:47:38.764 257256 INFO cinder.volume.manager [req-eb976e2d-47be-4d24-9a54-2cd15fa3c111 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Deleted volume successfully. >2018-08-24 03:47:38.998 257031 INFO cinder.volume.manager [req-8ed8e911-49bd-497a-ba19-f26a3b67c522 63d58f318a704228b5936e4e0d28ac67 471871d4cf794b49b4465e5797b632c2 - default default] Deleted volume successfully. >2018-08-24 03:47:43.673 257256 INFO cinder.volume.flows.manager.create_volume [req-4ccf1689-6d4c-479d-91d6-6075eecfd9b0 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Volume 84171567-7f77-4f2e-b805-650049cb9cce: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-84171567-7f77-4f2e-b805-650049cb9cce'} >2018-08-24 03:47:44.200 257256 INFO cinder.volume.flows.manager.create_volume [req-4ccf1689-6d4c-479d-91d6-6075eecfd9b0 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Volume volume-84171567-7f77-4f2e-b805-650049cb9cce (84171567-7f77-4f2e-b805-650049cb9cce): created successfully >2018-08-24 03:47:44.203 257256 INFO cinder.volume.manager [req-4ccf1689-6d4c-479d-91d6-6075eecfd9b0 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Created volume successfully. >2018-08-24 03:47:47.350 257256 INFO cinder.volume.manager [req-bdff382e-0bb8-4aac-ba5c-ae36421a76ef 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Create snapshot completed successfully >2018-08-24 03:47:50.014 257256 INFO cinder.volume.manager [req-3f2a36d2-f292-4769-9ade-213a0f6bd6bc 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Create snapshot completed successfully >2018-08-24 03:47:52.547 257256 INFO cinder.volume.manager [req-7e6c19a3-5949-4412-95f2-69505283ec8d 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Create snapshot completed successfully >2018-08-24 03:47:58.469 257256 INFO cinder.volume.manager [req-0791151a-0aae-49bb-8b76-7d199351a35c 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Delete snapshot completed successfully. >2018-08-24 03:47:58.880 257256 INFO cinder.volume.manager [req-7b002544-818b-439d-96e2-8b967d0ee849 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Delete snapshot completed successfully. >2018-08-24 03:47:58.988 257256 INFO cinder.volume.manager [req-5a394d9e-f000-4650-968d-8063423a1324 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Delete snapshot completed successfully. >2018-08-24 03:48:00.313 257256 INFO cinder.volume.manager [req-b3727a55-b4f1-4b85-a2df-c13222d8d4d8 90daca0ef2084a96a3e059c1389740aa 3b3622dabd3e453a918d210930a55628 - default default] Deleted volume successfully. >2018-08-24 03:48:05.514 257256 INFO cinder.volume.flows.manager.create_volume [req-71670a26-c5c1-4428-b4ce-d026b4fdca7f 82bd5200a9624689a64a903514f79703 af69596922114d0191eaf4622e716645 - default default] Volume 2af01d69-ec70-4535-9f8a-6d5cc2544ba0: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-2af01d69-ec70-4535-9f8a-6d5cc2544ba0'} >2018-08-24 03:48:06.036 257256 INFO cinder.volume.flows.manager.create_volume [req-71670a26-c5c1-4428-b4ce-d026b4fdca7f 82bd5200a9624689a64a903514f79703 af69596922114d0191eaf4622e716645 - default default] Volume volume-2af01d69-ec70-4535-9f8a-6d5cc2544ba0 (2af01d69-ec70-4535-9f8a-6d5cc2544ba0): created successfully >2018-08-24 03:48:06.040 257256 INFO cinder.volume.manager [req-71670a26-c5c1-4428-b4ce-d026b4fdca7f 82bd5200a9624689a64a903514f79703 af69596922114d0191eaf4622e716645 - default default] Created volume successfully. >2018-08-24 03:48:08.699 257256 INFO cinder.volume.manager [req-00c0916f-9c9d-43b7-a7e8-ee2be6aa7948 82bd5200a9624689a64a903514f79703 af69596922114d0191eaf4622e716645 - default default] Create snapshot completed successfully >2018-08-24 03:48:11.272 257256 INFO cinder.volume.manager [req-66272e01-ea5d-499b-ad38-a6813159566a 82bd5200a9624689a64a903514f79703 af69596922114d0191eaf4622e716645 - default default] Delete snapshot completed successfully. >2018-08-24 03:48:12.761 257256 INFO cinder.volume.manager [req-812987a1-bd77-451b-b424-3a388dc7fa9f 82bd5200a9624689a64a903514f79703 af69596922114d0191eaf4622e716645 - default default] Deleted volume successfully. >2018-08-24 03:48:34.786 257256 INFO cinder.volume.flows.manager.create_volume [req-f4d0b374-aa48-43fa-882b-6a9e19104d38 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume 3b690afe-80f5-4254-9f3c-74d50149c8e7: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-3b690afe-80f5-4254-9f3c-74d50149c8e7', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab547f81d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-24 03:48:35.623 257256 INFO cinder.image.image_utils [req-f4d0b374-aa48-43fa-882b-6a9e19104d38 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-24 03:48:50.455 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 03:48:56.610 257256 INFO cinder.image.image_utils [req-f4d0b374-aa48-43fa-882b-6a9e19104d38 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Converted 39.00 MB image at 21.97 MB/s >2018-08-24 03:48:59.546 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f4d0b374-aa48-43fa-882b-6a9e19104d38 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 03:49:00.019 257256 INFO cinder.volume.flows.manager.create_volume [req-f4d0b374-aa48-43fa-882b-6a9e19104d38 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume volume-3b690afe-80f5-4254-9f3c-74d50149c8e7 (3b690afe-80f5-4254-9f3c-74d50149c8e7): created successfully >2018-08-24 03:49:00.023 257256 INFO cinder.volume.manager [req-f4d0b374-aa48-43fa-882b-6a9e19104d38 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Created volume successfully. >2018-08-24 03:49:00.968 257256 INFO cinder.volume.flows.manager.create_volume [req-3f514983-efd9-436e-b00d-568fa63dc2c3 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume ac39ff15-03e0-410a-ba7b-d4d01775545d: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ac39ff15-03e0-410a-ba7b-d4d01775545d', 'source_volid': '3b690afe-80f5-4254-9f3c-74d50149c8e7', 'source_volstatus': u'available'} >2018-08-24 03:49:03.576 257256 INFO cinder.volume.flows.manager.create_volume [req-3f514983-efd9-436e-b00d-568fa63dc2c3 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume volume-ac39ff15-03e0-410a-ba7b-d4d01775545d (ac39ff15-03e0-410a-ba7b-d4d01775545d): created successfully >2018-08-24 03:49:03.580 257256 INFO cinder.volume.manager [req-3f514983-efd9-436e-b00d-568fa63dc2c3 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Created volume successfully. >2018-08-24 03:49:04.835 257031 INFO cinder.volume.flows.manager.create_volume [req-bc803cc1-3284-4edd-bb0c-21c8f07f7888 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume 3b102bae-ee64-4701-92c7-317a6e504878: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3b102bae-ee64-4701-92c7-317a6e504878'} >2018-08-24 03:49:05.757 257031 INFO cinder.volume.flows.manager.create_volume [req-bc803cc1-3284-4edd-bb0c-21c8f07f7888 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume volume-3b102bae-ee64-4701-92c7-317a6e504878 (3b102bae-ee64-4701-92c7-317a6e504878): created successfully >2018-08-24 03:49:05.762 257031 INFO cinder.volume.manager [req-bc803cc1-3284-4edd-bb0c-21c8f07f7888 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Created volume successfully. >2018-08-24 03:49:06.234 257031 INFO cinder.volume.flows.manager.create_volume [req-9171f93f-80e3-424d-a7ca-8283e12e3595 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume 43107357-4181-4e27-99ce-4151648095a6: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-43107357-4181-4e27-99ce-4151648095a6', 'source_volid': '3b102bae-ee64-4701-92c7-317a6e504878', 'source_volstatus': u'available'} >2018-08-24 03:49:30.230 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9171f93f-80e3-424d-a7ca-8283e12e3595 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Unable to get remote copy information for volume volume-43107357-4181-4e27-99ce-4151648095a6 >2018-08-24 03:49:30.540 257031 INFO cinder.volume.flows.manager.create_volume [req-9171f93f-80e3-424d-a7ca-8283e12e3595 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Volume volume-43107357-4181-4e27-99ce-4151648095a6 (43107357-4181-4e27-99ce-4151648095a6): created successfully >2018-08-24 03:49:30.543 257031 INFO cinder.volume.manager [req-9171f93f-80e3-424d-a7ca-8283e12e3595 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Created volume successfully. >2018-08-24 03:49:34.722 257256 INFO cinder.volume.manager [req-3a7a8810-9203-46fe-b716-ace0e4ed7885 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Deleted volume successfully. >2018-08-24 03:49:34.753 257256 INFO cinder.volume.manager [req-5ee5ce18-e87d-4558-8f96-00baf720cbb1 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Deleted volume successfully. >2018-08-24 03:49:34.905 257031 INFO cinder.volume.manager [req-dd2a19eb-1e32-4099-ada8-bdf60f3c1910 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Deleted volume successfully. >2018-08-24 03:49:35.063 257031 INFO cinder.volume.manager [req-eeea62ca-8a92-4652-9d05-79434c3f6214 03b31a20f9b54191b0f8e9a2c6a3a3be 9623d9b098b04fbc89ffeeaad22d62c5 - default default] Deleted volume successfully. >2018-08-24 03:49:39.691 257256 INFO cinder.volume.flows.manager.create_volume [req-86407aac-1006-4697-bb06-5a3ceb3be742 45e78abd75454168827fafbd455baa88 6637984d697743b4b62e4d65241a73fe - default default] Volume c42f3008-2810-4592-9403-4a1c8c655c80: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-c42f3008-2810-4592-9403-4a1c8c655c80'} >2018-08-24 03:49:40.217 257256 INFO cinder.volume.flows.manager.create_volume [req-86407aac-1006-4697-bb06-5a3ceb3be742 45e78abd75454168827fafbd455baa88 6637984d697743b4b62e4d65241a73fe - default default] Volume volume-c42f3008-2810-4592-9403-4a1c8c655c80 (c42f3008-2810-4592-9403-4a1c8c655c80): created successfully >2018-08-24 03:49:40.221 257256 INFO cinder.volume.manager [req-86407aac-1006-4697-bb06-5a3ceb3be742 45e78abd75454168827fafbd455baa88 6637984d697743b4b62e4d65241a73fe - default default] Created volume successfully. >2018-08-24 03:49:42.870 257256 INFO cinder.volume.manager [req-33b3d29b-cf14-46e9-9415-92a2290a60dd 45e78abd75454168827fafbd455baa88 6637984d697743b4b62e4d65241a73fe - default default] Deleted volume successfully. >2018-08-24 03:50:06.249 257256 INFO cinder.volume.flows.manager.create_volume [req-1522c45e-7e26-4ec9-8095-6f6f2608bb5b 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] Volume 5e18de38-ce76-46c4-a3e3-4aba91fc4b25: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5e18de38-ce76-46c4-a3e3-4aba91fc4b25'} >2018-08-24 03:50:06.783 257256 INFO cinder.volume.flows.manager.create_volume [req-1522c45e-7e26-4ec9-8095-6f6f2608bb5b 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] Volume volume-5e18de38-ce76-46c4-a3e3-4aba91fc4b25 (5e18de38-ce76-46c4-a3e3-4aba91fc4b25): created successfully >2018-08-24 03:50:06.787 257256 INFO cinder.volume.manager [req-1522c45e-7e26-4ec9-8095-6f6f2608bb5b 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] Created volume successfully. >2018-08-24 03:50:09.591 257256 INFO cinder.volume.manager [req-8a589a50-f0ff-415e-bec7-3bee00e168b7 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] Deleted volume successfully. >2018-08-24 03:50:10.376 257256 INFO cinder.volume.flows.manager.create_volume [req-6ccf488e-eab2-48b8-82b3-8ca0fde6454c 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] Volume 185fee9c-5a43-4b41-b247-f5425dce3765: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-185fee9c-5a43-4b41-b247-f5425dce3765'} >2018-08-24 03:50:10.901 257256 INFO cinder.volume.flows.manager.create_volume [req-6ccf488e-eab2-48b8-82b3-8ca0fde6454c 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] Volume volume-185fee9c-5a43-4b41-b247-f5425dce3765 (185fee9c-5a43-4b41-b247-f5425dce3765): created successfully >2018-08-24 03:50:10.905 257256 INFO cinder.volume.manager [req-6ccf488e-eab2-48b8-82b3-8ca0fde6454c 705eff7e753e45aa8be3ec631d2ade24 76efeda557d04e3492d7e8141107f450 - default default] Created volume successfully. >2018-08-24 03:50:12.548 257256 INFO cinder.volume.manager [req-6e6dacb2-762b-491f-9a4a-2e6d3a2532ff 762da80f37424010aaa33ef724728825 1255dc34b7dd41949ccd386e32225787 - default default] Transfer volume completed successfully. >2018-08-24 03:50:14.916 257256 INFO cinder.volume.manager [req-7ffbd73d-0f00-45c0-9fc7-2be61bdc5750 e6b7bac954de48a0befd7784faa958b3 d946fa8deb704af5b45fa0f4aedbb124 - default default] Deleted volume successfully. >2018-08-24 03:50:24.884 257256 INFO cinder.volume.flows.manager.create_volume [req-c51496fa-9108-45ad-bb62-16aed38d856b 69366f435800454382a45c2314f3fd71 b6398ad3873a4990b35a3b8eff6a3671 - default default] Volume 79030596-628b-45b1-bf3d-10375fb16a8f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-79030596-628b-45b1-bf3d-10375fb16a8f'} >2018-08-24 03:50:25.409 257256 INFO cinder.volume.flows.manager.create_volume [req-c51496fa-9108-45ad-bb62-16aed38d856b 69366f435800454382a45c2314f3fd71 b6398ad3873a4990b35a3b8eff6a3671 - default default] Volume volume-79030596-628b-45b1-bf3d-10375fb16a8f (79030596-628b-45b1-bf3d-10375fb16a8f): created successfully >2018-08-24 03:50:25.413 257256 INFO cinder.volume.manager [req-c51496fa-9108-45ad-bb62-16aed38d856b 69366f435800454382a45c2314f3fd71 b6398ad3873a4990b35a3b8eff6a3671 - default default] Created volume successfully. >2018-08-24 03:50:28.692 257256 INFO cinder.volume.manager [req-fdf7ca21-e2bc-45f5-85c4-fcbb895a753d 69366f435800454382a45c2314f3fd71 b6398ad3873a4990b35a3b8eff6a3671 - default default] Deleted volume successfully. >2018-08-24 03:50:36.341 257256 INFO cinder.volume.flows.manager.create_volume [req-26a22595-5b88-4737-925c-124d564a3f7c 8d5cf235f3c347b6832517cf6a69bf0a bd8b01b6bf594cb1b68a35153b7a30e3 - default default] Volume 9d744a97-9e3b-4522-b41d-8d88a67cbbdb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9d744a97-9e3b-4522-b41d-8d88a67cbbdb'} >2018-08-24 03:50:38.852 257256 INFO cinder.volume.flows.manager.create_volume [req-26a22595-5b88-4737-925c-124d564a3f7c 8d5cf235f3c347b6832517cf6a69bf0a bd8b01b6bf594cb1b68a35153b7a30e3 - default default] Volume volume-9d744a97-9e3b-4522-b41d-8d88a67cbbdb (9d744a97-9e3b-4522-b41d-8d88a67cbbdb): created successfully >2018-08-24 03:50:38.856 257256 INFO cinder.volume.manager [req-26a22595-5b88-4737-925c-124d564a3f7c 8d5cf235f3c347b6832517cf6a69bf0a bd8b01b6bf594cb1b68a35153b7a30e3 - default default] Created volume successfully. >2018-08-24 03:50:41.044 257256 INFO cinder.volume.manager [req-e56b36e7-43d7-4567-a4e3-ffa0a5081fec 8d5cf235f3c347b6832517cf6a69bf0a bd8b01b6bf594cb1b68a35153b7a30e3 - default default] Create snapshot completed successfully >2018-08-24 03:50:45.324 257256 INFO cinder.volume.manager [req-ddfaa1e7-b9fa-45de-ab3b-1db06f8caa47 8d5cf235f3c347b6832517cf6a69bf0a bd8b01b6bf594cb1b68a35153b7a30e3 - default default] Delete snapshot completed successfully. >2018-08-24 03:50:47.079 257256 INFO cinder.volume.manager [req-56272a5b-4938-429c-81be-ae3f1993d191 8d5cf235f3c347b6832517cf6a69bf0a bd8b01b6bf594cb1b68a35153b7a30e3 - default default] Deleted volume successfully. >2018-08-24 05:35:32.745 257031 INFO cinder.volume.flows.manager.create_volume [req-95e35e39-4c47-4e61-a4b0-dd66833203b1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume 27f45d41-1cc2-4a9f-8af7-49de9acada2e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-27f45d41-1cc2-4a9f-8af7-49de9acada2e'} >2018-08-24 05:35:33.682 257031 INFO cinder.volume.flows.manager.create_volume [req-95e35e39-4c47-4e61-a4b0-dd66833203b1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume volume-27f45d41-1cc2-4a9f-8af7-49de9acada2e (27f45d41-1cc2-4a9f-8af7-49de9acada2e): created successfully >2018-08-24 05:35:33.687 257031 INFO cinder.volume.manager [req-95e35e39-4c47-4e61-a4b0-dd66833203b1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Created volume successfully. >2018-08-24 05:38:45.430 257031 INFO cinder.volume.manager [req-46326135-3a1a-4c0f-8d8d-3b37d25cb3c1 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Initialize volume connection completed successfully. >2018-08-24 05:38:51.391 257031 INFO cinder.volume.manager [req-f07ce4b5-c09e-4cfd-bd02-ff4ca732ae48 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Attaching volume 27f45d41-1cc2-4a9f-8af7-49de9acada2e to instance 9cdcfd6a-d4be-4d2c-aae6-abca865d29ea at mountpoint /dev/vdb on host None. >2018-08-24 05:38:51.468 257031 INFO cinder.volume.manager [req-f07ce4b5-c09e-4cfd-bd02-ff4ca732ae48 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Attach volume completed successfully. >2018-08-24 06:27:04.190 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e0f1c7bb-6547-494a-980f-9cce5e79ee29 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Unable to get remote copy information for volume volume-27f45d41-1cc2-4a9f-8af7-49de9acada2e >2018-08-24 06:27:05.009 257031 INFO cinder.compute.nova [req-e0f1c7bb-6547-494a-980f-9cce5e79ee29 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Nova event response: {u'status': u'completed', u'tag': u'27f45d41-1cc2-4a9f-8af7-49de9acada2e', u'name': u'volume-extended', u'server_uuid': u'9cdcfd6a-d4be-4d2c-aae6-abca865d29ea', u'code': 200}. >2018-08-24 06:27:05.035 257031 INFO cinder.volume.manager [req-e0f1c7bb-6547-494a-980f-9cce5e79ee29 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Extend volume completed successfully. >2018-08-24 06:35:47.619 257256 INFO cinder.volume.flows.manager.create_volume [req-ec8a6218-e6f0-4905-932c-d6be09f42926 e618fc81d42148419cdedb7ca02724e0 c285a1e69815457caf190b79c547d494 - default default] Volume b025adcb-66d5-4b7c-9287-65af072dbd27: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b025adcb-66d5-4b7c-9287-65af072dbd27'} >2018-08-24 06:35:48.141 257256 INFO cinder.volume.flows.manager.create_volume [req-ec8a6218-e6f0-4905-932c-d6be09f42926 e618fc81d42148419cdedb7ca02724e0 c285a1e69815457caf190b79c547d494 - default default] Volume volume-b025adcb-66d5-4b7c-9287-65af072dbd27 (b025adcb-66d5-4b7c-9287-65af072dbd27): created successfully >2018-08-24 06:35:48.145 257256 INFO cinder.volume.manager [req-ec8a6218-e6f0-4905-932c-d6be09f42926 e618fc81d42148419cdedb7ca02724e0 c285a1e69815457caf190b79c547d494 - default default] Created volume successfully. >2018-08-24 06:35:52.586 257256 INFO cinder.volume.manager [req-85849d45-d33b-487c-9957-7ba72bc8f9ad e618fc81d42148419cdedb7ca02724e0 c285a1e69815457caf190b79c547d494 - default default] Deleted volume successfully. >2018-08-24 06:36:01.549 257256 INFO cinder.volume.flows.manager.create_volume [req-7f0b29dc-3b01-4e03-92f9-bbaa2cf37d11 36ec81ee2f30498a804ac81ab7d3165f 7f7a034c70784126955db1bbfdacf574 - default default] Volume f2bb115c-a039-467d-983a-dbe8455eecaf: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f2bb115c-a039-467d-983a-dbe8455eecaf'} >2018-08-24 06:36:02.090 257256 INFO cinder.volume.flows.manager.create_volume [req-7f0b29dc-3b01-4e03-92f9-bbaa2cf37d11 36ec81ee2f30498a804ac81ab7d3165f 7f7a034c70784126955db1bbfdacf574 - default default] Volume volume-f2bb115c-a039-467d-983a-dbe8455eecaf (f2bb115c-a039-467d-983a-dbe8455eecaf): created successfully >2018-08-24 06:36:02.095 257256 INFO cinder.volume.manager [req-7f0b29dc-3b01-4e03-92f9-bbaa2cf37d11 36ec81ee2f30498a804ac81ab7d3165f 7f7a034c70784126955db1bbfdacf574 - default default] Created volume successfully. >2018-08-24 06:36:05.921 257256 WARNING cinder.quota [req-2e010f1e-9f47-4f42-b5dc-bc0c03f5dcbe 36ec81ee2f30498a804ac81ab7d3165f 7f7a034c70784126955db1bbfdacf574 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1087855098 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1087855098, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:36:05.922 257256 WARNING cinder.quota [req-2e010f1e-9f47-4f42-b5dc-bc0c03f5dcbe 36ec81ee2f30498a804ac81ab7d3165f 7f7a034c70784126955db1bbfdacf574 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1087855098 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1087855098, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:36:06.208 257256 INFO cinder.volume.manager [req-2e010f1e-9f47-4f42-b5dc-bc0c03f5dcbe 36ec81ee2f30498a804ac81ab7d3165f 7f7a034c70784126955db1bbfdacf574 - default default] Deleted volume successfully. >2018-08-24 06:36:15.477 257256 INFO cinder.volume.flows.manager.create_volume [req-e8bd5010-14d4-46f3-8ca6-b03f1e39d088 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] Volume f50a02b6-2da1-4a60-9720-63c7078ca46a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f50a02b6-2da1-4a60-9720-63c7078ca46a'} >2018-08-24 06:36:16.020 257256 INFO cinder.volume.flows.manager.create_volume [req-e8bd5010-14d4-46f3-8ca6-b03f1e39d088 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] Volume volume-f50a02b6-2da1-4a60-9720-63c7078ca46a (f50a02b6-2da1-4a60-9720-63c7078ca46a): created successfully >2018-08-24 06:36:16.024 257256 INFO cinder.volume.manager [req-e8bd5010-14d4-46f3-8ca6-b03f1e39d088 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] Created volume successfully. >2018-08-24 06:36:17.635 257256 INFO cinder.volume.manager [req-fc4e0c7c-71bb-43b1-8717-e5d5b5d54cdf 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] Retype volume completed successfully. >2018-08-24 06:36:21.429 257256 WARNING cinder.quota [req-50092f9f-a427-428c-b5c5-13177f8a830c 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-559578297 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-559578297, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:36:21.431 257256 WARNING cinder.quota [req-50092f9f-a427-428c-b5c5-13177f8a830c 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-559578297 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-559578297, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:36:21.690 257256 INFO cinder.volume.manager [req-50092f9f-a427-428c-b5c5-13177f8a830c 8e6907849f4847ceb30cf9adbcc4fd7a 5c0b2f3b11d94cda9e446f8d2db5bccf - default default] Deleted volume successfully. >2018-08-24 06:39:05.127 257256 INFO cinder.volume.flows.manager.create_volume [req-c1cbeaee-83b1-475b-a83d-b52971c1dda8 57d9fe57a88e4e2c9c2161865a405acf 27fb2da81cf649ccb90a4aee448237ba - default default] Volume 2872b1be-aec7-4025-b510-17a71f3e9f43: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2872b1be-aec7-4025-b510-17a71f3e9f43'} >2018-08-24 06:39:05.649 257256 INFO cinder.volume.flows.manager.create_volume [req-c1cbeaee-83b1-475b-a83d-b52971c1dda8 57d9fe57a88e4e2c9c2161865a405acf 27fb2da81cf649ccb90a4aee448237ba - default default] Volume volume-2872b1be-aec7-4025-b510-17a71f3e9f43 (2872b1be-aec7-4025-b510-17a71f3e9f43): created successfully >2018-08-24 06:39:05.653 257256 INFO cinder.volume.manager [req-c1cbeaee-83b1-475b-a83d-b52971c1dda8 57d9fe57a88e4e2c9c2161865a405acf 27fb2da81cf649ccb90a4aee448237ba - default default] Created volume successfully. >2018-08-24 06:39:08.229 257256 INFO cinder.volume.manager [req-d5201748-8a53-4eb2-b984-ef8242feb0bb 57d9fe57a88e4e2c9c2161865a405acf 27fb2da81cf649ccb90a4aee448237ba - default default] Deleted volume successfully. >2018-08-24 06:39:17.285 257256 INFO cinder.volume.flows.manager.create_volume [req-d64d3384-245c-43cd-a08f-9e8ffe45bfe4 b46050689aae4f4592e5a898776846fe 2a97bf1e8a1d44f88c9684c82aa5c8e9 - default default] Volume 1db98261-698f-4ea6-aebf-ae14928cea89: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1db98261-698f-4ea6-aebf-ae14928cea89'} >2018-08-24 06:39:17.830 257256 INFO cinder.volume.flows.manager.create_volume [req-d64d3384-245c-43cd-a08f-9e8ffe45bfe4 b46050689aae4f4592e5a898776846fe 2a97bf1e8a1d44f88c9684c82aa5c8e9 - default default] Volume volume-1db98261-698f-4ea6-aebf-ae14928cea89 (1db98261-698f-4ea6-aebf-ae14928cea89): created successfully >2018-08-24 06:39:17.833 257256 INFO cinder.volume.manager [req-d64d3384-245c-43cd-a08f-9e8ffe45bfe4 b46050689aae4f4592e5a898776846fe 2a97bf1e8a1d44f88c9684c82aa5c8e9 - default default] Created volume successfully. >2018-08-24 06:39:21.298 257256 WARNING cinder.quota [req-726d9da3-c150-4c72-93a6-dc84ffed8570 b46050689aae4f4592e5a898776846fe 2a97bf1e8a1d44f88c9684c82aa5c8e9 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-836037326 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-836037326, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:39:21.299 257256 WARNING cinder.quota [req-726d9da3-c150-4c72-93a6-dc84ffed8570 b46050689aae4f4592e5a898776846fe 2a97bf1e8a1d44f88c9684c82aa5c8e9 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-836037326 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-836037326, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:39:21.572 257256 INFO cinder.volume.manager [req-726d9da3-c150-4c72-93a6-dc84ffed8570 b46050689aae4f4592e5a898776846fe 2a97bf1e8a1d44f88c9684c82aa5c8e9 - default default] Deleted volume successfully. >2018-08-24 06:39:30.220 257256 INFO cinder.volume.flows.manager.create_volume [req-89022a2c-e0e3-4fd0-bfad-5b9530cf7174 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] Volume 23ddc4f9-9456-4802-b59d-614525eab265: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-23ddc4f9-9456-4802-b59d-614525eab265'} >2018-08-24 06:39:30.790 257256 INFO cinder.volume.flows.manager.create_volume [req-89022a2c-e0e3-4fd0-bfad-5b9530cf7174 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] Volume volume-23ddc4f9-9456-4802-b59d-614525eab265 (23ddc4f9-9456-4802-b59d-614525eab265): created successfully >2018-08-24 06:39:30.794 257256 INFO cinder.volume.manager [req-89022a2c-e0e3-4fd0-bfad-5b9530cf7174 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] Created volume successfully. >2018-08-24 06:39:32.612 257256 INFO cinder.volume.manager [req-708e206c-abc4-488b-a68b-e91bcc3ea332 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] Retype volume completed successfully. >2018-08-24 06:39:36.163 257256 WARNING cinder.quota [req-c28f1ee7-0e75-48c4-a54c-eca8231d47b7 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1210372809 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1210372809, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:39:36.164 257256 WARNING cinder.quota [req-c28f1ee7-0e75-48c4-a54c-eca8231d47b7 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1210372809 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1210372809, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:39:36.419 257256 INFO cinder.volume.manager [req-c28f1ee7-0e75-48c4-a54c-eca8231d47b7 b88ad1ca6b3f47bf8affa00d4793497c 055ee43a5f6442b78e429d3513caf963 - default default] Deleted volume successfully. >2018-08-24 06:40:16.215 257256 INFO cinder.volume.flows.manager.create_volume [req-23fa15e3-ff7e-44da-95e1-48b01477216f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume 5d4ac9e1-ecfa-464b-8834-8aa527eee741: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5d4ac9e1-ecfa-464b-8834-8aa527eee741'} >2018-08-24 06:40:16.739 257256 INFO cinder.volume.flows.manager.create_volume [req-23fa15e3-ff7e-44da-95e1-48b01477216f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume volume-5d4ac9e1-ecfa-464b-8834-8aa527eee741 (5d4ac9e1-ecfa-464b-8834-8aa527eee741): created successfully >2018-08-24 06:40:16.742 257256 INFO cinder.volume.manager [req-23fa15e3-ff7e-44da-95e1-48b01477216f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Created volume successfully. >2018-08-24 06:40:17.914 257256 INFO cinder.volume.manager [req-e1d02abe-e996-439b-ba4a-ab96cce0951d 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Attaching volume 5d4ac9e1-ecfa-464b-8834-8aa527eee741 to instance e239389f-bfd7-4365-a06a-5574014bf11a at mountpoint /dev/vdb on host None. >2018-08-24 06:40:17.982 257256 INFO cinder.volume.manager [req-e1d02abe-e996-439b-ba4a-ab96cce0951d 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Attach volume completed successfully. >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager [req-b1c002d2-34b2-4fc3-9798-28ef5743d167 06fe76feda24489d88945a35f5b0165c 8e1ec68746ed440a9b98b10544792475 - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager force=force) >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-24 06:40:19.249 257256 ERROR cinder.volume.manager >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server [req-b1c002d2-34b2-4fc3-9798-28ef5743d167 06fe76feda24489d88945a35f5b0165c 8e1ec68746ed440a9b98b10544792475 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-24 06:40:19.252 257256 ERROR oslo_messaging.rpc.server >2018-08-24 06:40:19.380 257256 INFO cinder.volume.manager [req-2959af85-45ee-40fe-9ad6-651eb8cffd06 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Detaching volume 5d4ac9e1-ecfa-464b-8834-8aa527eee741 from instance e239389f-bfd7-4365-a06a-5574014bf11a. >2018-08-24 06:40:19.462 257256 INFO cinder.volume.manager [req-2959af85-45ee-40fe-9ad6-651eb8cffd06 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Detach volume completed successfully. >2018-08-24 06:40:21.528 257031 INFO cinder.volume.flows.manager.create_volume [req-a460d589-f9f5-48f0-a637-afad516f977a 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume 73ce52d2-f500-43e5-bd86-231a4bcc0f58: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-73ce52d2-f500-43e5-bd86-231a4bcc0f58'} >2018-08-24 06:40:22.055 257031 INFO cinder.volume.flows.manager.create_volume [req-a460d589-f9f5-48f0-a637-afad516f977a 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume volume-73ce52d2-f500-43e5-bd86-231a4bcc0f58 (73ce52d2-f500-43e5-bd86-231a4bcc0f58): created successfully >2018-08-24 06:40:22.058 257031 INFO cinder.volume.manager [req-a460d589-f9f5-48f0-a637-afad516f977a 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Created volume successfully. >2018-08-24 06:40:24.404 257031 INFO cinder.volume.manager [req-c0798cc2-b206-4839-90e8-f1be0ed8ca50 06fe76feda24489d88945a35f5b0165c 8e1ec68746ed440a9b98b10544792475 - default default] Deleted volume successfully. >2018-08-24 06:40:24.732 257256 INFO cinder.volume.flows.manager.create_volume [req-0180e291-5eb7-428b-bf80-244ba640bb9f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume 3adc26ab-ebbd-4b72-a5a2-a503269f43c6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3adc26ab-ebbd-4b72-a5a2-a503269f43c6'} >2018-08-24 06:40:25.272 257256 INFO cinder.volume.flows.manager.create_volume [req-0180e291-5eb7-428b-bf80-244ba640bb9f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume volume-3adc26ab-ebbd-4b72-a5a2-a503269f43c6 (3adc26ab-ebbd-4b72-a5a2-a503269f43c6): created successfully >2018-08-24 06:40:25.276 257256 INFO cinder.volume.manager [req-0180e291-5eb7-428b-bf80-244ba640bb9f 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Created volume successfully. >2018-08-24 06:40:27.618 257256 INFO cinder.volume.manager [req-db3f1d33-21b8-4631-8e24-cc7adaee78f0 06fe76feda24489d88945a35f5b0165c 8e1ec68746ed440a9b98b10544792475 - default default] Deleted volume successfully. >2018-08-24 06:40:28.583 257256 INFO cinder.volume.flows.manager.create_volume [req-40c4b709-097d-4a0a-9d65-9cba985fdf3e 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume ad2fcfc0-5552-4526-8a9b-e2c2f4283e56: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ad2fcfc0-5552-4526-8a9b-e2c2f4283e56'} >2018-08-24 06:40:29.111 257256 INFO cinder.volume.flows.manager.create_volume [req-40c4b709-097d-4a0a-9d65-9cba985fdf3e 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume volume-ad2fcfc0-5552-4526-8a9b-e2c2f4283e56 (ad2fcfc0-5552-4526-8a9b-e2c2f4283e56): created successfully >2018-08-24 06:40:29.115 257256 INFO cinder.volume.manager [req-40c4b709-097d-4a0a-9d65-9cba985fdf3e 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Created volume successfully. >2018-08-24 06:40:31.058 257256 INFO cinder.volume.manager [req-051ad394-2cb4-4df6-9348-06dc2273014f 06fe76feda24489d88945a35f5b0165c 8e1ec68746ed440a9b98b10544792475 - default default] Deleted volume successfully. >2018-08-24 06:40:31.165 257031 INFO cinder.volume.flows.manager.create_volume [req-9fc4b555-4e2e-43a9-bfc2-66ef339883ca 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume 7c5ea101-f3f2-479d-bb7c-02330e6a98d5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7c5ea101-f3f2-479d-bb7c-02330e6a98d5'} >2018-08-24 06:40:31.709 257031 INFO cinder.volume.flows.manager.create_volume [req-9fc4b555-4e2e-43a9-bfc2-66ef339883ca 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume volume-7c5ea101-f3f2-479d-bb7c-02330e6a98d5 (7c5ea101-f3f2-479d-bb7c-02330e6a98d5): created successfully >2018-08-24 06:40:31.712 257031 INFO cinder.volume.manager [req-9fc4b555-4e2e-43a9-bfc2-66ef339883ca 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Created volume successfully. >2018-08-24 06:40:33.863 257031 INFO cinder.volume.manager [req-35664f1b-ca86-4173-8cbb-db482d22ae16 06fe76feda24489d88945a35f5b0165c 8e1ec68746ed440a9b98b10544792475 - default default] Deleted volume successfully. >2018-08-24 06:40:34.254 257256 INFO cinder.volume.flows.manager.create_volume [req-99c2cdeb-b977-4317-86f0-07971689bcf5 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume 528d9333-fae7-495d-9da4-b004b95484b7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-528d9333-fae7-495d-9da4-b004b95484b7'} >2018-08-24 06:40:34.777 257256 INFO cinder.volume.flows.manager.create_volume [req-99c2cdeb-b977-4317-86f0-07971689bcf5 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Volume volume-528d9333-fae7-495d-9da4-b004b95484b7 (528d9333-fae7-495d-9da4-b004b95484b7): created successfully >2018-08-24 06:40:34.781 257256 INFO cinder.volume.manager [req-99c2cdeb-b977-4317-86f0-07971689bcf5 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Created volume successfully. >2018-08-24 06:40:38.103 257256 INFO cinder.volume.manager [req-951300a4-434b-4f84-85df-d8ba05ef9f02 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Deleted volume successfully. >2018-08-24 06:40:39.068 257256 INFO cinder.volume.manager [req-be63f4f3-bff2-499a-87ce-1730b73146e3 194e41720d5b4d81a2e7179cd941fccd 347c1c1bafb0489799981ea601d73a3a - default default] Deleted volume successfully. >2018-08-24 06:40:46.036 257256 INFO cinder.volume.flows.manager.create_volume [req-dcefd152-7d30-4b35-bf68-4ff915850786 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Volume 8d6af455-daac-4b5c-b71b-2d0f500de856: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8d6af455-daac-4b5c-b71b-2d0f500de856'} >2018-08-24 06:40:46.569 257256 INFO cinder.volume.flows.manager.create_volume [req-dcefd152-7d30-4b35-bf68-4ff915850786 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Volume volume-8d6af455-daac-4b5c-b71b-2d0f500de856 (8d6af455-daac-4b5c-b71b-2d0f500de856): created successfully >2018-08-24 06:40:46.573 257256 INFO cinder.volume.manager [req-dcefd152-7d30-4b35-bf68-4ff915850786 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Created volume successfully. >2018-08-24 06:40:48.276 257256 INFO cinder.volume.flows.manager.create_volume [req-448e550a-12e0-4f77-98c7-618a5f38d69c 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Volume 133f6b71-50af-4e4d-afc0-49180f7d8191: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-133f6b71-50af-4e4d-afc0-49180f7d8191'} >2018-08-24 06:40:48.808 257256 INFO cinder.volume.flows.manager.create_volume [req-448e550a-12e0-4f77-98c7-618a5f38d69c 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Volume volume-133f6b71-50af-4e4d-afc0-49180f7d8191 (133f6b71-50af-4e4d-afc0-49180f7d8191): created successfully >2018-08-24 06:40:48.812 257256 INFO cinder.volume.manager [req-448e550a-12e0-4f77-98c7-618a5f38d69c 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Created volume successfully. >2018-08-24 06:40:49.892 257031 INFO cinder.volume.flows.manager.create_volume [req-d46e91e2-ab57-463b-95b9-681803c3cae5 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Volume 29f6f591-fd3e-4a1f-b92c-e39b7448a438: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-29f6f591-fd3e-4a1f-b92c-e39b7448a438'} >2018-08-24 06:40:50.412 257031 INFO cinder.volume.flows.manager.create_volume [req-d46e91e2-ab57-463b-95b9-681803c3cae5 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Volume volume-29f6f591-fd3e-4a1f-b92c-e39b7448a438 (29f6f591-fd3e-4a1f-b92c-e39b7448a438): created successfully >2018-08-24 06:40:50.416 257031 INFO cinder.volume.manager [req-d46e91e2-ab57-463b-95b9-681803c3cae5 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Created volume successfully. >2018-08-24 06:40:51.962 257256 INFO cinder.volume.flows.manager.create_volume [req-8782ce71-d8d4-4353-9395-4f545eca6e89 42d7b40c2b9c400b9d4e8bf13cd9d213 e9a4dc48803f45e4aff75086fd405b89 - default default] Volume 5401f29f-c717-4efa-8682-17cec399425c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5401f29f-c717-4efa-8682-17cec399425c'} >2018-08-24 06:40:52.485 257256 INFO cinder.volume.flows.manager.create_volume [req-8782ce71-d8d4-4353-9395-4f545eca6e89 42d7b40c2b9c400b9d4e8bf13cd9d213 e9a4dc48803f45e4aff75086fd405b89 - default default] Volume volume-5401f29f-c717-4efa-8682-17cec399425c (5401f29f-c717-4efa-8682-17cec399425c): created successfully >2018-08-24 06:40:52.489 257256 INFO cinder.volume.manager [req-8782ce71-d8d4-4353-9395-4f545eca6e89 42d7b40c2b9c400b9d4e8bf13cd9d213 e9a4dc48803f45e4aff75086fd405b89 - default default] Created volume successfully. >2018-08-24 06:40:59.738 257256 INFO cinder.volume.manager [req-2e1274ea-9f36-41da-8c8e-772e0d070c34 42d7b40c2b9c400b9d4e8bf13cd9d213 e9a4dc48803f45e4aff75086fd405b89 - default default] Deleted volume successfully. >2018-08-24 06:40:59.757 257256 INFO cinder.volume.manager [req-2617d7f2-48e0-4eb1-8643-8bb9eb489652 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Deleted volume successfully. >2018-08-24 06:40:59.779 257256 INFO cinder.volume.manager [req-f255d04e-df0e-46b3-8cc1-3fe413f0ec2b 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Deleted volume successfully. >2018-08-24 06:40:59.971 257031 INFO cinder.volume.manager [req-62674848-b0ac-4abb-8e28-ae05b235e2b3 0cbf8fede1e8469694d5a50559042894 d19005ae72d747b8a173b193a1f96d0a - default default] Deleted volume successfully. >2018-08-24 06:41:05.887 257256 INFO cinder.volume.flows.manager.create_volume [req-937db51c-c96b-4f83-b08c-201bff62ad7d 8950b7f571df4eac983385a135fa6702 f3607fae8c1b47e79a403d398a4e8113 - default default] Volume c3fbb5d0-ee5d-4db3-9b7b-249e6f8e7139: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c3fbb5d0-ee5d-4db3-9b7b-249e6f8e7139'} >2018-08-24 06:41:06.422 257256 INFO cinder.volume.flows.manager.create_volume [req-937db51c-c96b-4f83-b08c-201bff62ad7d 8950b7f571df4eac983385a135fa6702 f3607fae8c1b47e79a403d398a4e8113 - default default] Volume volume-c3fbb5d0-ee5d-4db3-9b7b-249e6f8e7139 (c3fbb5d0-ee5d-4db3-9b7b-249e6f8e7139): created successfully >2018-08-24 06:41:06.430 257256 INFO cinder.volume.manager [req-937db51c-c96b-4f83-b08c-201bff62ad7d 8950b7f571df4eac983385a135fa6702 f3607fae8c1b47e79a403d398a4e8113 - default default] Created volume successfully. >2018-08-24 06:41:11.224 257256 INFO cinder.volume.manager [req-e823d5db-ff13-4dfc-be4d-313509773e07 8950b7f571df4eac983385a135fa6702 f3607fae8c1b47e79a403d398a4e8113 - default default] Deleted volume successfully. >2018-08-24 06:41:18.121 257256 INFO cinder.volume.flows.manager.create_volume [req-848d9bac-37cc-4049-9742-584ddb7c3927 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] Volume edefc06b-9ab5-4ac8-ad86-4812d14324ce: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-edefc06b-9ab5-4ac8-ad86-4812d14324ce'} >2018-08-24 06:41:18.662 257256 INFO cinder.volume.flows.manager.create_volume [req-848d9bac-37cc-4049-9742-584ddb7c3927 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] Volume volume-edefc06b-9ab5-4ac8-ad86-4812d14324ce (edefc06b-9ab5-4ac8-ad86-4812d14324ce): created successfully >2018-08-24 06:41:18.666 257256 INFO cinder.volume.manager [req-848d9bac-37cc-4049-9742-584ddb7c3927 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] Created volume successfully. >2018-08-24 06:41:20.742 257256 INFO cinder.volume.manager [req-8201fbee-30ff-4f36-8416-c536984f2d93 246fb4f0fbd44404904b2b2f70bdd385 23cf4395bf0342a684615602af69cb90 - default default] Transfer volume completed successfully. >2018-08-24 06:41:22.974 257256 INFO cinder.volume.manager [req-8d49e568-7376-4d43-ba16-04f00054a12d 4c79eb9e8f88448980a5b083eb6735b1 a73ab98d459741e697aadc3158bb64f5 - default default] Deleted volume successfully. >2018-08-24 06:41:23.922 257256 INFO cinder.volume.flows.manager.create_volume [req-85af4da6-b8dd-420d-af81-000ccc444ed2 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] Volume ec665747-b849-4239-be11-9e8eae0bea8e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ec665747-b849-4239-be11-9e8eae0bea8e'} >2018-08-24 06:41:24.463 257256 INFO cinder.volume.flows.manager.create_volume [req-85af4da6-b8dd-420d-af81-000ccc444ed2 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] Volume volume-ec665747-b849-4239-be11-9e8eae0bea8e (ec665747-b849-4239-be11-9e8eae0bea8e): created successfully >2018-08-24 06:41:24.467 257256 INFO cinder.volume.manager [req-85af4da6-b8dd-420d-af81-000ccc444ed2 10c0ee47dcbe48e983ecf0e66dcb34c1 94433386434f4f7e8211ecbc77c71f1e - default default] Created volume successfully. >2018-08-24 06:41:27.723 257256 INFO cinder.volume.manager [req-3d87b8cf-ca80-4f2f-b43a-1e4f428e0bcb 4c79eb9e8f88448980a5b083eb6735b1 a73ab98d459741e697aadc3158bb64f5 - default default] Deleted volume successfully. >2018-08-24 06:41:34.891 257256 INFO cinder.volume.flows.manager.create_volume [req-0f2e5648-22cc-4457-9f76-a30985bcc425 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Volume 822b6648-6360-4066-93c6-adca715cb48a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-822b6648-6360-4066-93c6-adca715cb48a'} >2018-08-24 06:41:35.665 257256 INFO cinder.volume.flows.manager.create_volume [req-0f2e5648-22cc-4457-9f76-a30985bcc425 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Volume volume-822b6648-6360-4066-93c6-adca715cb48a (822b6648-6360-4066-93c6-adca715cb48a): created successfully >2018-08-24 06:41:35.669 257256 INFO cinder.volume.manager [req-0f2e5648-22cc-4457-9f76-a30985bcc425 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Created volume successfully. >2018-08-24 06:41:42.257 257256 INFO cinder.volume.manager [req-328a08ee-d494-497c-aecd-adf2cc5bda61 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Attaching volume 822b6648-6360-4066-93c6-adca715cb48a to instance 86335041-19f0-4cd2-9f98-f2170eece9a6 at mountpoint /dev/vdb on host None. >2018-08-24 06:41:42.328 257256 INFO cinder.volume.manager [req-328a08ee-d494-497c-aecd-adf2cc5bda61 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Attach volume completed successfully. >2018-08-24 06:41:42.740 257256 INFO cinder.volume.manager [req-63538f5e-8804-4acd-8f55-41405bbb7e97 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Detaching volume 822b6648-6360-4066-93c6-adca715cb48a from instance 86335041-19f0-4cd2-9f98-f2170eece9a6. >2018-08-24 06:41:42.827 257256 INFO cinder.volume.manager [req-63538f5e-8804-4acd-8f55-41405bbb7e97 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Detach volume completed successfully. >2018-08-24 06:41:50.527 257256 INFO cinder.volume.manager [req-92be3c8c-589e-4348-b91e-2b0c86084b06 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Attaching volume 822b6648-6360-4066-93c6-adca715cb48a to instance a357b7ee-d1d3-4971-a5d6-fa0944c80a4b at mountpoint /dev/vdb on host None. >2018-08-24 06:41:50.597 257256 INFO cinder.volume.manager [req-92be3c8c-589e-4348-b91e-2b0c86084b06 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Attach volume completed successfully. >2018-08-24 06:41:51.111 257256 INFO cinder.volume.manager [req-3540a109-99bd-4907-81cb-bc56fc618d6b dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Detaching volume 822b6648-6360-4066-93c6-adca715cb48a from instance a357b7ee-d1d3-4971-a5d6-fa0944c80a4b. >2018-08-24 06:41:51.196 257256 INFO cinder.volume.manager [req-3540a109-99bd-4907-81cb-bc56fc618d6b dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Detach volume completed successfully. >2018-08-24 06:42:08.509 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 06:42:28.167 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-7d14ff3a-e376-4507-9e8b-35e98ebd9280 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 06:42:28.620 257256 INFO cinder.volume.manager [req-7d14ff3a-e376-4507-9e8b-35e98ebd9280 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Copy volume to image completed successfully. >2018-08-24 06:42:31.081 257256 INFO cinder.volume.manager [req-3f365876-ab6e-4ba6-8e77-2f620c84a8a3 dee57a9880434fab9c5fc77c86187de7 5b58644470b2444aa817a5ff03403d2f - default default] Deleted volume successfully. >2018-08-24 06:42:37.494 257256 INFO cinder.volume.flows.manager.create_volume [req-a7b7ecad-0fb8-429f-ba1e-b6e17a624063 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Volume 3caf0a10-8690-4a61-81f4-1228e0506ec3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3caf0a10-8690-4a61-81f4-1228e0506ec3'} >2018-08-24 06:42:38.024 257256 INFO cinder.volume.flows.manager.create_volume [req-a7b7ecad-0fb8-429f-ba1e-b6e17a624063 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Volume volume-3caf0a10-8690-4a61-81f4-1228e0506ec3 (3caf0a10-8690-4a61-81f4-1228e0506ec3): created successfully >2018-08-24 06:42:38.028 257256 INFO cinder.volume.manager [req-a7b7ecad-0fb8-429f-ba1e-b6e17a624063 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Created volume successfully. >2018-08-24 06:42:49.579 257256 INFO cinder.volume.manager [req-cbc8278e-5040-46eb-901c-ad5bffd56221 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Initialize volume connection completed successfully. >2018-08-24 06:42:54.757 257256 INFO cinder.volume.manager [req-34924e0f-8dc8-4bc1-9434-3c5bf22c5460 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Attaching volume 3caf0a10-8690-4a61-81f4-1228e0506ec3 to instance 16d56428-909d-4962-8d04-9592a15d0096 at mountpoint /dev/vdb on host None. >2018-08-24 06:42:54.829 257256 INFO cinder.volume.manager [req-34924e0f-8dc8-4bc1-9434-3c5bf22c5460 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Attach volume completed successfully. >2018-08-24 06:42:56.412 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-2fd1c223-7eaa-4bd9-b5e5-46afca111743 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Unable to get remote copy information for volume volume-3caf0a10-8690-4a61-81f4-1228e0506ec3 >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova [req-2fd1c223-7eaa-4bd9-b5e5-46afca111743 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Failed to notify nova on events: [{'tag': '3caf0a10-8690-4a61-81f4-1228e0506ec3', 'name': 'volume-extended', 'server_uuid': '16d56428-909d-4962-8d04-9592a15d0096'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-7bcb7092-20b8-469d-bae0-bbdacb480f0e) >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova return_raw=True) >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-7bcb7092-20b8-469d-bae0-bbdacb480f0e) >2018-08-24 06:42:57.164 257256 ERROR cinder.compute.nova >2018-08-24 06:42:57.190 257256 INFO cinder.volume.manager [req-2fd1c223-7eaa-4bd9-b5e5-46afca111743 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Extend volume completed successfully. >2018-08-24 06:48:08.680 257256 INFO cinder.volume.manager [req-59defb99-df04-4094-8c15-19d5feb7d54b 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Terminate volume connection completed successfully. >2018-08-24 06:48:09.087 257256 INFO cinder.volume.manager [req-077df438-7518-40be-8eee-4a56597f142e 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Detaching volume 3caf0a10-8690-4a61-81f4-1228e0506ec3 from instance 16d56428-909d-4962-8d04-9592a15d0096. >2018-08-24 06:48:09.176 257256 INFO cinder.volume.manager [req-077df438-7518-40be-8eee-4a56597f142e 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Detach volume completed successfully. >2018-08-24 06:48:14.013 257256 INFO cinder.volume.manager [req-85bcd5cb-8a8c-4225-ae98-481db81e423c 25fcd3fddc4b4bd9a10df12ea4243fc5 368292f770e6459eb6ee5ba95f019cb3 - default default] Deleted volume successfully. >2018-08-24 06:48:19.984 257256 INFO cinder.volume.flows.manager.create_volume [req-9640bc78-d1b0-400e-a83e-ce26a2773c41 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] Volume 83a27c24-3a1f-4780-b8a3-b8dee6e7989a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-83a27c24-3a1f-4780-b8a3-b8dee6e7989a'} >2018-08-24 06:48:20.529 257256 INFO cinder.volume.flows.manager.create_volume [req-9640bc78-d1b0-400e-a83e-ce26a2773c41 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] Volume volume-83a27c24-3a1f-4780-b8a3-b8dee6e7989a (83a27c24-3a1f-4780-b8a3-b8dee6e7989a): created successfully >2018-08-24 06:48:20.533 257256 INFO cinder.volume.manager [req-9640bc78-d1b0-400e-a83e-ce26a2773c41 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] Created volume successfully. >2018-08-24 06:48:21.900 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-297f4194-5722-455f-931d-2ebe6c861198 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] Unable to get remote copy information for volume volume-83a27c24-3a1f-4780-b8a3-b8dee6e7989a >2018-08-24 06:48:22.201 257256 INFO cinder.volume.manager [req-297f4194-5722-455f-931d-2ebe6c861198 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] Extend volume completed successfully. >2018-08-24 06:48:24.362 257256 INFO cinder.volume.manager [req-c3371b86-fd69-45cc-b901-9bfe5107cbd9 03f791603bff4d8ba5c1ec1b89af4332 ecf5d5bdd12c4dcab54667dfb449583c - default default] Deleted volume successfully. >2018-08-24 06:48:29.161 257256 INFO cinder.volume.flows.manager.create_volume [req-705ac65b-82fc-4cff-a0a7-878365032b9b b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume 6ec1b09e-9b45-4e9b-9e73-8890132f687d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6ec1b09e-9b45-4e9b-9e73-8890132f687d'} >2018-08-24 06:48:29.712 257256 INFO cinder.volume.flows.manager.create_volume [req-705ac65b-82fc-4cff-a0a7-878365032b9b b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume volume-6ec1b09e-9b45-4e9b-9e73-8890132f687d (6ec1b09e-9b45-4e9b-9e73-8890132f687d): created successfully >2018-08-24 06:48:29.716 257256 INFO cinder.volume.manager [req-705ac65b-82fc-4cff-a0a7-878365032b9b b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Created volume successfully. >2018-08-24 06:48:31.998 257256 INFO cinder.volume.flows.manager.create_volume [req-aa3a5477-9504-45a4-8cfa-6f7089cfdb1a b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume 82604068-54b6-4fe1-9240-20d5ee71c6f2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-82604068-54b6-4fe1-9240-20d5ee71c6f2'} >2018-08-24 06:48:32.938 257256 INFO cinder.volume.flows.manager.create_volume [req-aa3a5477-9504-45a4-8cfa-6f7089cfdb1a b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume volume-82604068-54b6-4fe1-9240-20d5ee71c6f2 (82604068-54b6-4fe1-9240-20d5ee71c6f2): created successfully >2018-08-24 06:48:32.942 257256 INFO cinder.volume.manager [req-aa3a5477-9504-45a4-8cfa-6f7089cfdb1a b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Created volume successfully. >2018-08-24 06:48:35.459 257256 INFO cinder.volume.manager [req-3037a010-e69f-476c-9508-8ac9f92e93cd b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Deleted volume successfully. >2018-08-24 06:48:37.874 257256 INFO cinder.volume.manager [req-d0859c41-b346-4159-a05b-d0554b5f78b1 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Deleted volume successfully. >2018-08-24 06:48:38.855 257256 INFO cinder.volume.flows.manager.create_volume [req-8bbdb37f-04c5-40b0-ab8e-a7acd2dd8caa b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume b1b79874-2f05-4130-b0ac-367915bf1e43: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b1b79874-2f05-4130-b0ac-367915bf1e43'} >2018-08-24 06:48:39.783 257256 INFO cinder.volume.flows.manager.create_volume [req-8bbdb37f-04c5-40b0-ab8e-a7acd2dd8caa b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume volume-b1b79874-2f05-4130-b0ac-367915bf1e43 (b1b79874-2f05-4130-b0ac-367915bf1e43): created successfully >2018-08-24 06:48:39.787 257256 INFO cinder.volume.manager [req-8bbdb37f-04c5-40b0-ab8e-a7acd2dd8caa b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Created volume successfully. >2018-08-24 06:48:41.210 257256 INFO cinder.volume.flows.manager.create_volume [req-537205a0-450f-4fed-bf67-b2d4567fdff1 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume e205ddf1-641b-42fe-87b8-6039cccaf6f7: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e205ddf1-641b-42fe-87b8-6039cccaf6f7', 'source_volid': 'b1b79874-2f05-4130-b0ac-367915bf1e43', 'source_volstatus': u'available'} >2018-08-24 06:48:46.727 257256 INFO cinder.volume.flows.manager.create_volume [req-537205a0-450f-4fed-bf67-b2d4567fdff1 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume volume-e205ddf1-641b-42fe-87b8-6039cccaf6f7 (e205ddf1-641b-42fe-87b8-6039cccaf6f7): created successfully >2018-08-24 06:48:46.731 257256 INFO cinder.volume.manager [req-537205a0-450f-4fed-bf67-b2d4567fdff1 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Created volume successfully. >2018-08-24 06:48:47.483 257256 INFO cinder.volume.flows.manager.create_volume [req-fc4d00b5-b010-4928-8f4f-065cc798bc80 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume e668f5ad-791e-4979-8168-cd9a5430a98d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e668f5ad-791e-4979-8168-cd9a5430a98d'} >2018-08-24 06:48:48.320 257256 INFO cinder.volume.flows.manager.create_volume [req-fc4d00b5-b010-4928-8f4f-065cc798bc80 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume volume-e668f5ad-791e-4979-8168-cd9a5430a98d (e668f5ad-791e-4979-8168-cd9a5430a98d): created successfully >2018-08-24 06:48:48.323 257256 INFO cinder.volume.manager [req-fc4d00b5-b010-4928-8f4f-065cc798bc80 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Created volume successfully. >2018-08-24 06:48:50.760 257256 INFO cinder.volume.manager [req-14dbbd9b-e312-47d5-b194-b85fcd32924c b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Deleted volume successfully. >2018-08-24 06:48:55.819 257256 INFO cinder.volume.manager [req-4c44692a-e4f2-4846-a806-b7882bc6892c b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Deleted volume successfully. >2018-08-24 06:48:57.977 257256 INFO cinder.volume.flows.manager.create_volume [req-7bd48503-a972-497f-8b52-a6fd0a9870f2 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume ea00ad2d-0d4d-4656-8855-d7fa7d87f56c: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-ea00ad2d-0d4d-4656-8855-d7fa7d87f56c', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab547a8550>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-24 06:48:58.845 257256 INFO cinder.image.image_utils [req-7bd48503-a972-497f-8b52-a6fd0a9870f2 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-24 06:49:14.065 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 06:49:18.732 257256 INFO cinder.image.image_utils [req-7bd48503-a972-497f-8b52-a6fd0a9870f2 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-24 06:49:21.223 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-7bd48503-a972-497f-8b52-a6fd0a9870f2 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 06:49:21.758 257256 INFO cinder.volume.flows.manager.create_volume [req-7bd48503-a972-497f-8b52-a6fd0a9870f2 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume volume-ea00ad2d-0d4d-4656-8855-d7fa7d87f56c (ea00ad2d-0d4d-4656-8855-d7fa7d87f56c): created successfully >2018-08-24 06:49:21.762 257256 INFO cinder.volume.manager [req-7bd48503-a972-497f-8b52-a6fd0a9870f2 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Created volume successfully. >2018-08-24 06:49:23.126 257256 INFO cinder.volume.flows.manager.create_volume [req-cb2698ce-741b-4dec-ad5e-f5c2af98850e b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume f9c452bc-d18c-43c3-8d76-4aca0b041fe6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f9c452bc-d18c-43c3-8d76-4aca0b041fe6'} >2018-08-24 06:49:23.661 257256 INFO cinder.volume.flows.manager.create_volume [req-cb2698ce-741b-4dec-ad5e-f5c2af98850e b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Volume volume-f9c452bc-d18c-43c3-8d76-4aca0b041fe6 (f9c452bc-d18c-43c3-8d76-4aca0b041fe6): created successfully >2018-08-24 06:49:23.665 257256 INFO cinder.volume.manager [req-cb2698ce-741b-4dec-ad5e-f5c2af98850e b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Created volume successfully. >2018-08-24 06:49:25.410 257256 INFO cinder.volume.manager [req-8ae14aaf-2d5c-4e81-9a42-7b08847975c5 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Deleted volume successfully. >2018-08-24 06:49:26.656 257256 INFO cinder.volume.manager [req-46df3401-8a89-499d-89ac-f5aebdd669b8 b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Deleted volume successfully. >2018-08-24 06:49:27.952 257256 INFO cinder.volume.manager [req-e9f70ef8-2c5d-4ec6-b3bb-6ca92eaf84bd b2168c24f05c404babd7fb9ee5c50f73 b60e6f29f3954028a7784002c6fb2084 - default default] Deleted volume successfully. >2018-08-24 06:49:36.483 257256 INFO cinder.volume.flows.manager.create_volume [req-43e85c8c-ce1d-48a5-99ba-b3cdc71a3737 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Volume c3e8c287-79b3-4187-839a-91b949265522: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c3e8c287-79b3-4187-839a-91b949265522'} >2018-08-24 06:49:37.011 257256 INFO cinder.volume.flows.manager.create_volume [req-43e85c8c-ce1d-48a5-99ba-b3cdc71a3737 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Volume volume-c3e8c287-79b3-4187-839a-91b949265522 (c3e8c287-79b3-4187-839a-91b949265522): created successfully >2018-08-24 06:49:37.015 257256 INFO cinder.volume.manager [req-43e85c8c-ce1d-48a5-99ba-b3cdc71a3737 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Created volume successfully. >2018-08-24 06:49:38.466 257256 INFO cinder.volume.flows.manager.create_volume [req-f0ed7902-a3a3-479d-9a36-1d93022bbf3a 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Volume 8dd68e7e-699a-4628-8090-51e2abfbcc9d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8dd68e7e-699a-4628-8090-51e2abfbcc9d'} >2018-08-24 06:49:38.985 257256 INFO cinder.volume.flows.manager.create_volume [req-f0ed7902-a3a3-479d-9a36-1d93022bbf3a 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Volume volume-8dd68e7e-699a-4628-8090-51e2abfbcc9d (8dd68e7e-699a-4628-8090-51e2abfbcc9d): created successfully >2018-08-24 06:49:38.989 257256 INFO cinder.volume.manager [req-f0ed7902-a3a3-479d-9a36-1d93022bbf3a 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Created volume successfully. >2018-08-24 06:49:40.289 257256 INFO cinder.volume.flows.manager.create_volume [req-c6332155-abc2-49e9-b2fd-e79ec84a0f4d 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Volume cbf06fd2-d217-4ca1-84a6-26b6ef70e254: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cbf06fd2-d217-4ca1-84a6-26b6ef70e254'} >2018-08-24 06:49:40.812 257256 INFO cinder.volume.flows.manager.create_volume [req-c6332155-abc2-49e9-b2fd-e79ec84a0f4d 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Volume volume-cbf06fd2-d217-4ca1-84a6-26b6ef70e254 (cbf06fd2-d217-4ca1-84a6-26b6ef70e254): created successfully >2018-08-24 06:49:40.817 257256 INFO cinder.volume.manager [req-c6332155-abc2-49e9-b2fd-e79ec84a0f4d 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Created volume successfully. >2018-08-24 06:49:47.222 257256 INFO cinder.volume.manager [req-7f0231b6-2d2c-43d9-b50f-3049023b7bf3 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Deleted volume successfully. >2018-08-24 06:49:47.249 257256 INFO cinder.volume.manager [req-0addbe0c-e355-4570-b0da-7070fd13cd30 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Deleted volume successfully. >2018-08-24 06:49:47.467 257256 INFO cinder.volume.manager [req-716c6214-e815-40f0-958e-e46fa12c9338 8d715c53e33b43ae9558667dc7b2f95b 5d21d94458af4e898c47964269201e1e - default default] Deleted volume successfully. >2018-08-24 06:49:52.339 257256 INFO cinder.volume.flows.manager.create_volume [req-520fa953-6616-4859-aa0a-136dfea17616 a083600daf824d20a84f3bda79a21bc6 284ecc02c2eb4ac0ba1b925507803385 - default default] Volume e0d812ee-b851-4a50-a939-73965af545d1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e0d812ee-b851-4a50-a939-73965af545d1'} >2018-08-24 06:49:52.865 257256 INFO cinder.volume.flows.manager.create_volume [req-520fa953-6616-4859-aa0a-136dfea17616 a083600daf824d20a84f3bda79a21bc6 284ecc02c2eb4ac0ba1b925507803385 - default default] Volume volume-e0d812ee-b851-4a50-a939-73965af545d1 (e0d812ee-b851-4a50-a939-73965af545d1): created successfully >2018-08-24 06:49:52.869 257256 INFO cinder.volume.manager [req-520fa953-6616-4859-aa0a-136dfea17616 a083600daf824d20a84f3bda79a21bc6 284ecc02c2eb4ac0ba1b925507803385 - default default] Created volume successfully. >2018-08-24 06:50:12.943 257256 INFO cinder.volume.manager [req-ac3e2277-5949-40a3-8c94-bf7cafadc083 a083600daf824d20a84f3bda79a21bc6 284ecc02c2eb4ac0ba1b925507803385 - default default] Deleted volume successfully. >2018-08-24 06:50:17.342 257256 INFO cinder.volume.flows.manager.create_volume [req-208f3e43-18de-4c58-b15b-7ff22b93a3dc 44bdbf13719a413ea06a6be3d3665124 cb8f6ac7b02141ca95444b87836ead00 - default default] Volume 47bc8754-8234-4f11-8a17-77f21a45a16d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-47bc8754-8234-4f11-8a17-77f21a45a16d'} >2018-08-24 06:50:17.868 257256 INFO cinder.volume.flows.manager.create_volume [req-208f3e43-18de-4c58-b15b-7ff22b93a3dc 44bdbf13719a413ea06a6be3d3665124 cb8f6ac7b02141ca95444b87836ead00 - default default] Volume volume-47bc8754-8234-4f11-8a17-77f21a45a16d (47bc8754-8234-4f11-8a17-77f21a45a16d): created successfully >2018-08-24 06:50:17.872 257256 INFO cinder.volume.manager [req-208f3e43-18de-4c58-b15b-7ff22b93a3dc 44bdbf13719a413ea06a6be3d3665124 cb8f6ac7b02141ca95444b87836ead00 - default default] Created volume successfully. >2018-08-24 06:50:20.497 257256 INFO cinder.volume.manager [req-6944cd27-66e3-4b92-aa0b-690055f12a35 44bdbf13719a413ea06a6be3d3665124 cb8f6ac7b02141ca95444b87836ead00 - default default] Deleted volume successfully. >2018-08-24 06:51:07.576 257256 INFO cinder.volume.flows.manager.create_volume [req-c818e4d7-5ca5-4642-8117-8e77583a1c19 42f4456ee09a481c813cc1d5395c46ad b13ffdba266a4ccfaa4a5c1552083540 - default default] Volume fa1e17f6-7ffe-4bf6-85d0-0bd21aa42443: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fa1e17f6-7ffe-4bf6-85d0-0bd21aa42443'} >2018-08-24 06:51:10.415 257256 INFO cinder.volume.flows.manager.create_volume [req-c818e4d7-5ca5-4642-8117-8e77583a1c19 42f4456ee09a481c813cc1d5395c46ad b13ffdba266a4ccfaa4a5c1552083540 - default default] Volume volume-fa1e17f6-7ffe-4bf6-85d0-0bd21aa42443 (fa1e17f6-7ffe-4bf6-85d0-0bd21aa42443): created successfully >2018-08-24 06:51:10.418 257256 INFO cinder.volume.manager [req-c818e4d7-5ca5-4642-8117-8e77583a1c19 42f4456ee09a481c813cc1d5395c46ad b13ffdba266a4ccfaa4a5c1552083540 - default default] Created volume successfully. >2018-08-24 06:51:13.967 257256 INFO cinder.volume.manager [req-a5d001cb-37d2-473c-a6b2-7a9292700a29 42f4456ee09a481c813cc1d5395c46ad b13ffdba266a4ccfaa4a5c1552083540 - default default] Deleted volume successfully. >2018-08-24 06:51:19.320 257256 INFO cinder.volume.flows.manager.create_volume [req-1d600d93-9aa4-4a39-8631-f3c67cc5c1a5 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Volume 1e1f6345-06ec-47ac-87a7-1552c065f23f: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-1e1f6345-06ec-47ac-87a7-1552c065f23f', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54780350>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-24 06:51:20.347 257256 INFO cinder.image.image_utils [req-1d600d93-9aa4-4a39-8631-f3c67cc5c1a5 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Image download 12.00 MB at 11.71 MB/s >2018-08-24 06:51:30.006 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 06:51:34.996 257256 INFO cinder.image.image_utils [req-1d600d93-9aa4-4a39-8631-f3c67cc5c1a5 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-24 06:51:37.023 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-1d600d93-9aa4-4a39-8631-f3c67cc5c1a5 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 06:51:37.371 257256 INFO cinder.volume.flows.manager.create_volume [req-1d600d93-9aa4-4a39-8631-f3c67cc5c1a5 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Volume volume-1e1f6345-06ec-47ac-87a7-1552c065f23f (1e1f6345-06ec-47ac-87a7-1552c065f23f): created successfully >2018-08-24 06:51:37.374 257256 INFO cinder.volume.manager [req-1d600d93-9aa4-4a39-8631-f3c67cc5c1a5 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Created volume successfully. >2018-08-24 06:51:38.791 257256 INFO cinder.volume.flows.manager.create_volume [req-40267cd3-be0d-4fc5-90a8-b4870fcfefbf b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Volume a21c2836-b2a0-4eba-ae19-e41acdc7f290: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a21c2836-b2a0-4eba-ae19-e41acdc7f290', 'source_volid': '1e1f6345-06ec-47ac-87a7-1552c065f23f', 'source_volstatus': u'available'} >2018-08-24 06:51:40.278 257256 INFO cinder.volume.flows.manager.create_volume [req-40267cd3-be0d-4fc5-90a8-b4870fcfefbf b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Volume volume-a21c2836-b2a0-4eba-ae19-e41acdc7f290 (a21c2836-b2a0-4eba-ae19-e41acdc7f290): created successfully >2018-08-24 06:51:40.282 257256 INFO cinder.volume.manager [req-40267cd3-be0d-4fc5-90a8-b4870fcfefbf b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Created volume successfully. >2018-08-24 06:51:43.824 257256 INFO cinder.volume.manager [req-a232528c-d73b-4a72-b5db-a8416cfa1ec9 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Deleted volume successfully. >2018-08-24 06:51:45.488 257256 INFO cinder.volume.manager [req-a4d85e32-b9d9-4de1-8724-9b662b245293 b24eb4893dd643ae839c43bcfd721a6c e1f6d49ae67646fd8630bb8aa71b449b - default default] Deleted volume successfully. >2018-08-24 06:51:50.844 257256 INFO cinder.volume.flows.manager.create_volume [req-ab75ac83-6b9c-43d3-820e-bce4576142aa be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Volume 3fd223ef-94c2-426c-9399-b37cb74e3b53: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3fd223ef-94c2-426c-9399-b37cb74e3b53'} >2018-08-24 06:51:51.368 257256 INFO cinder.volume.flows.manager.create_volume [req-ab75ac83-6b9c-43d3-820e-bce4576142aa be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Volume volume-3fd223ef-94c2-426c-9399-b37cb74e3b53 (3fd223ef-94c2-426c-9399-b37cb74e3b53): created successfully >2018-08-24 06:51:51.372 257256 INFO cinder.volume.manager [req-ab75ac83-6b9c-43d3-820e-bce4576142aa be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Created volume successfully. >2018-08-24 06:51:52.655 257031 INFO cinder.volume.flows.manager.create_volume [req-01b73275-7770-45f4-880a-83638e1ba7af be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Volume 8c817091-de34-480f-8a42-a929a002db7a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8c817091-de34-480f-8a42-a929a002db7a'} >2018-08-24 06:51:53.192 257031 INFO cinder.volume.flows.manager.create_volume [req-01b73275-7770-45f4-880a-83638e1ba7af be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Volume volume-8c817091-de34-480f-8a42-a929a002db7a (8c817091-de34-480f-8a42-a929a002db7a): created successfully >2018-08-24 06:51:53.196 257031 INFO cinder.volume.manager [req-01b73275-7770-45f4-880a-83638e1ba7af be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Created volume successfully. >2018-08-24 06:51:54.519 257256 INFO cinder.volume.flows.manager.create_volume [req-90f84fd8-6059-4e16-a28c-e36f128e429d be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Volume d5f212b9-4e46-4075-af0f-70dd2e4f83e6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d5f212b9-4e46-4075-af0f-70dd2e4f83e6'} >2018-08-24 06:51:55.129 257256 INFO cinder.volume.flows.manager.create_volume [req-90f84fd8-6059-4e16-a28c-e36f128e429d be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Volume volume-d5f212b9-4e46-4075-af0f-70dd2e4f83e6 (d5f212b9-4e46-4075-af0f-70dd2e4f83e6): created successfully >2018-08-24 06:51:55.133 257256 INFO cinder.volume.manager [req-90f84fd8-6059-4e16-a28c-e36f128e429d be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Created volume successfully. >2018-08-24 06:51:58.638 257256 INFO cinder.volume.manager [req-0cfc0cb3-f670-4a96-96c0-5f26c57fff43 be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Deleted volume successfully. >2018-08-24 06:51:58.888 257031 INFO cinder.volume.manager [req-9a860766-e180-4af5-8ccd-3138ceb55bf1 be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Deleted volume successfully. >2018-08-24 06:51:59.153 257256 INFO cinder.volume.manager [req-35d3e64c-7f59-41ac-a0cd-21d40e069d2a be28beea9d514b5b941a8787a474f5e7 0fbdfa7027244dd69353f485ac296ba0 - default default] Deleted volume successfully. >2018-08-24 06:52:19.532 257031 INFO cinder.volume.flows.manager.create_volume [req-c12a71ef-f681-4f02-ba9c-daa88eca32c6 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Volume 1a6610a4-8d0b-49d8-9ec8-1751443d4e64: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64'} >2018-08-24 06:52:20.092 257031 INFO cinder.volume.flows.manager.create_volume [req-c12a71ef-f681-4f02-ba9c-daa88eca32c6 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Volume volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 (1a6610a4-8d0b-49d8-9ec8-1751443d4e64): created successfully >2018-08-24 06:52:20.096 257031 INFO cinder.volume.manager [req-c12a71ef-f681-4f02-ba9c-daa88eca32c6 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Created volume successfully. >2018-08-24 06:52:22.119 257256 INFO cinder.volume.flows.manager.create_volume [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Volume c26fda01-3d73-4ab2-9790-94c3fc85157d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c26fda01-3d73-4ab2-9790-94c3fc85157d'} >2018-08-24 06:52:22.788 257256 INFO cinder.volume.flows.manager.create_volume [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Volume volume-c26fda01-3d73-4ab2-9790-94c3fc85157d (c26fda01-3d73-4ab2-9790-94c3fc85157d): created successfully >2018-08-24 06:52:22.792 257256 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Created volume successfully. >2018-08-24 06:52:30.029 257256 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Initialize volume connection completed successfully. >2018-08-24 06:52:30.418 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 06:52:35.362 257031 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Initialize volume connection completed successfully. >2018-08-24 06:52:35.748 257031 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 06:52:41.187 257031 INFO cinder.volume.utils [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Volume copy 1024.00 MB at 411.58 MB/s >2018-08-24 06:52:43.718 257256 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Terminate volume connection completed successfully. >2018-08-24 06:52:43.763 257256 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Remove volume export completed successfully. >2018-08-24 06:52:46.225 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 06:52:46.626 257031 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Terminate volume connection completed successfully. >2018-08-24 06:52:46.662 257031 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Remove volume export completed successfully. >2018-08-24 06:52:47.049 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 06:52:50.291 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Error running SSH command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 06:52:50.292 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 06:52:50.293 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Error running SSH command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 06:52:50.294 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64', '"volume-c26fda01-3d73-4ab2-9790-94c3fc85157d"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64 "volume-c26fda01-3d73-4ab2-9790-94c3fc85157d" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-24 06:52:50.294 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Unable to rename the logical volume for volume: 1a6610a4-8d0b-49d8-9ec8-1751443d4e64: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-1a6610a4-8d0b-49d8-9ec8-1751443d4e64', '"volume-c26fda01-3d73-4ab2-9790-94c3fc85157d"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-24 06:52:50.447 257031 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Complete-Migrate volume completed successfully. >2018-08-24 06:52:50.448 257031 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Migrate volume completed successfully. >2018-08-24 06:52:51.072 257031 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Retype volume completed successfully. >2018-08-24 06:52:52.384 257031 INFO cinder.volume.manager [req-1ed1e784-c6e3-42df-8074-888a08f337e5 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Deleted volume successfully. >2018-08-24 06:52:54.367 257256 WARNING cinder.quota [req-2b59c840-2531-4b2c-952a-4737bcb67a8b 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1355196425 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1355196425, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:52:54.369 257256 WARNING cinder.quota [req-2b59c840-2531-4b2c-952a-4737bcb67a8b 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1355196425 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1355196425, it is now deprecated. Please use the default quota class for default quota. >2018-08-24 06:52:54.849 257256 INFO cinder.volume.manager [req-2b59c840-2531-4b2c-952a-4737bcb67a8b 2daded66b66f4f3891d5c68126d8ee16 68e4372f91744f42a301af80e5f10f42 - default default] Deleted volume successfully. >2018-08-24 06:53:17.054 257256 INFO cinder.volume.flows.manager.create_volume [req-26a275d2-7293-4b2b-91e5-07b901d4d604 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Volume 8f599aad-0ba2-4aff-bf60-c9da00f56567: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8f599aad-0ba2-4aff-bf60-c9da00f56567'} >2018-08-24 06:53:17.898 257256 INFO cinder.volume.flows.manager.create_volume [req-26a275d2-7293-4b2b-91e5-07b901d4d604 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Volume volume-8f599aad-0ba2-4aff-bf60-c9da00f56567 (8f599aad-0ba2-4aff-bf60-c9da00f56567): created successfully >2018-08-24 06:53:17.902 257256 INFO cinder.volume.manager [req-26a275d2-7293-4b2b-91e5-07b901d4d604 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Created volume successfully. >2018-08-24 06:53:20.438 257256 INFO cinder.volume.manager [req-9a7784f0-78ca-4a6c-973e-f594cfe97cb0 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Create snapshot completed successfully >2018-08-24 06:53:23.879 257256 INFO cinder.volume.manager [req-bb63a79f-aa48-414c-b7cf-f9f6015a1e51 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Create snapshot completed successfully >2018-08-24 06:53:27.319 257256 INFO cinder.volume.manager [req-caacc69c-1d5f-40cc-bcc5-abe29ef71560 47416f2336ef4de0bea13b49f8907307 ebefe4a81e58447cbfa0f46da7de0407 - default default] Delete snapshot completed successfully. >2018-08-24 06:53:29.458 257256 INFO cinder.volume.manager [req-b7a55c72-a5cd-4de2-949e-2cc2a2aed0e5 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Create snapshot completed successfully >2018-08-24 06:53:32.944 257256 INFO cinder.volume.manager [req-89b0b09f-70da-427c-9c57-123c6998b960 47416f2336ef4de0bea13b49f8907307 ebefe4a81e58447cbfa0f46da7de0407 - default default] Delete snapshot completed successfully. >2018-08-24 06:53:36.058 257256 INFO cinder.volume.manager [req-237b1cfd-0a8d-45b7-894f-9cfc5c1d2bfd aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Create snapshot completed successfully >2018-08-24 06:53:38.981 257256 INFO cinder.volume.manager [req-d8781b19-a09c-455a-895d-7a98aa6eb898 47416f2336ef4de0bea13b49f8907307 ebefe4a81e58447cbfa0f46da7de0407 - default default] Delete snapshot completed successfully. >2018-08-24 06:53:41.437 257256 INFO cinder.volume.manager [req-eb3238f7-2807-4f08-be41-c85b10d991d6 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Create snapshot completed successfully >2018-08-24 06:53:44.121 257256 INFO cinder.volume.manager [req-d50d4a77-49c2-4383-a0e0-c4a56a26971a 47416f2336ef4de0bea13b49f8907307 ebefe4a81e58447cbfa0f46da7de0407 - default default] Delete snapshot completed successfully. >2018-08-24 06:53:47.010 257256 INFO cinder.volume.manager [req-0db55910-db79-43f2-9b01-3d073c2a4bbf aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Delete snapshot completed successfully. >2018-08-24 06:53:48.665 257256 INFO cinder.volume.manager [req-550b54c9-f29b-4b09-af58-babf50813df6 aaf1fd27ebce42d1830159ca0d83b5b3 8697fffbb6534b2a92bd236051b6c095 - default default] Deleted volume successfully. >2018-08-24 06:53:54.586 257256 INFO cinder.volume.flows.manager.create_volume [req-799475c3-0796-46e7-b511-6ee3d532e03b 75c0d7aaef724f92b09dd8ecd1845f90 7052d9d69bf44049a6ed64343f319cbd - default default] Volume b6a7c5f9-683f-4160-ae63-482a5edc8fd6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b6a7c5f9-683f-4160-ae63-482a5edc8fd6'} >2018-08-24 06:53:55.108 257256 INFO cinder.volume.flows.manager.create_volume [req-799475c3-0796-46e7-b511-6ee3d532e03b 75c0d7aaef724f92b09dd8ecd1845f90 7052d9d69bf44049a6ed64343f319cbd - default default] Volume volume-b6a7c5f9-683f-4160-ae63-482a5edc8fd6 (b6a7c5f9-683f-4160-ae63-482a5edc8fd6): created successfully >2018-08-24 06:53:55.112 257256 INFO cinder.volume.manager [req-799475c3-0796-46e7-b511-6ee3d532e03b 75c0d7aaef724f92b09dd8ecd1845f90 7052d9d69bf44049a6ed64343f319cbd - default default] Created volume successfully. >2018-08-24 06:53:58.220 257256 INFO cinder.volume.manager [req-51f4006c-91ea-4f30-8860-1795250ce241 75c0d7aaef724f92b09dd8ecd1845f90 7052d9d69bf44049a6ed64343f319cbd - default default] Create snapshot completed successfully >2018-08-24 06:54:03.335 257256 INFO cinder.volume.manager [req-a5536da2-d5af-42dc-b8ac-894aa54b6a06 75c0d7aaef724f92b09dd8ecd1845f90 7052d9d69bf44049a6ed64343f319cbd - default default] Delete snapshot completed successfully. >2018-08-24 06:54:05.058 257256 INFO cinder.volume.manager [req-bacae068-2d32-491b-a3c6-7a1ec4041805 75c0d7aaef724f92b09dd8ecd1845f90 7052d9d69bf44049a6ed64343f319cbd - default default] Deleted volume successfully. >2018-08-24 06:54:09.405 257256 INFO cinder.volume.flows.manager.create_volume [req-998364e5-5fd6-4677-bb2c-659c78b5d242 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Volume a5f1900b-2305-4f02-aea7-192800a80c67: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a5f1900b-2305-4f02-aea7-192800a80c67'} >2018-08-24 06:54:10.102 257256 INFO cinder.volume.flows.manager.create_volume [req-998364e5-5fd6-4677-bb2c-659c78b5d242 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Volume volume-a5f1900b-2305-4f02-aea7-192800a80c67 (a5f1900b-2305-4f02-aea7-192800a80c67): created successfully >2018-08-24 06:54:10.106 257256 INFO cinder.volume.manager [req-998364e5-5fd6-4677-bb2c-659c78b5d242 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Created volume successfully. >2018-08-24 06:54:12.797 257256 INFO cinder.volume.manager [req-689bb52b-acaf-459e-a844-0dde5e264587 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:54:27.480 257256 INFO cinder.volume.manager [req-05fa2a02-503f-4ce2-92fc-100189c470e5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Initialize volume connection completed successfully. >2018-08-24 06:54:50.560 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-05fa2a02-503f-4ce2-92fc-100189c470e5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 06:54:50.835 257256 INFO cinder.volume.manager [req-05fa2a02-503f-4ce2-92fc-100189c470e5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Terminate volume connection completed successfully. >2018-08-24 06:54:50.880 257256 INFO cinder.volume.manager [req-05fa2a02-503f-4ce2-92fc-100189c470e5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Remove volume export completed successfully. >2018-08-24 06:54:52.031 257256 INFO cinder.volume.manager [req-05fa2a02-503f-4ce2-92fc-100189c470e5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Deleted volume successfully. >2018-08-24 06:55:03.971 257256 INFO cinder.volume.manager [req-99d6f65e-3332-4b25-bf2b-64cc66f1e5c6 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Initialize volume connection completed successfully. >2018-08-24 06:55:08.673 257256 INFO cinder.volume.manager [req-de2e6e44-eac4-4e8c-94e9-e351ec352fe7 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Attaching volume a5f1900b-2305-4f02-aea7-192800a80c67 to instance bd98fdf5-e480-49ca-97ba-0e17070f4fae at mountpoint /dev/vdb on host None. >2018-08-24 06:55:08.749 257256 INFO cinder.volume.manager [req-de2e6e44-eac4-4e8c-94e9-e351ec352fe7 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Attach volume completed successfully. >2018-08-24 06:55:11.347 257256 INFO cinder.volume.manager [req-c6b8dbef-6483-4bdd-832b-55d30510d052 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:55:13.707 257256 INFO cinder.volume.manager [req-3d7722b0-5193-40d8-b1a8-51d50ff76881 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:55:16.192 257256 INFO cinder.volume.manager [req-8c844e8d-b9bb-4df3-8f31-c8cb27a59347 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:55:18.837 257256 INFO cinder.volume.manager [req-2e57703b-2e2c-4298-b7d8-a0cedc5106f2 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:55:20.926 257256 INFO cinder.volume.manager [req-b16ef52d-eaf9-4200-a025-15be9c8239b6 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:55:23.520 257256 INFO cinder.volume.manager [req-e6ea9a5e-80b8-42e3-bf2c-19083afe3ea7 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:55:34.894 257256 INFO cinder.volume.manager [req-e223a0ce-5d2f-462b-b24f-f36e10fda817 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Terminate volume connection completed successfully. >2018-08-24 06:55:35.018 257256 INFO cinder.volume.manager [req-6abe6d6e-c4b7-4743-b4d3-809722b95fc9 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Detaching volume a5f1900b-2305-4f02-aea7-192800a80c67 from instance bd98fdf5-e480-49ca-97ba-0e17070f4fae. >2018-08-24 06:55:35.102 257256 INFO cinder.volume.manager [req-6abe6d6e-c4b7-4743-b4d3-809722b95fc9 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Detach volume completed successfully. >2018-08-24 06:55:39.515 257256 INFO cinder.volume.manager [req-06b520a2-4c52-413b-ae76-b76787bb2342 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:55:46.111 257256 INFO cinder.volume.manager [req-0c1488e9-a8ca-4e23-86dd-e0d050fbd755 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:55:48.451 257256 INFO cinder.volume.manager [req-b1010d76-8fc2-4557-b1c6-2223b7b641a7 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:55:59.745 257256 INFO cinder.volume.manager [req-517f3549-e82e-4293-89db-b324eb028ab5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Initialize volume connection completed successfully. >2018-08-24 06:56:04.516 257256 INFO cinder.volume.manager [req-bfc476be-e405-401d-8320-7f19bb76e0b7 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Attaching volume a5f1900b-2305-4f02-aea7-192800a80c67 to instance b6ad3537-2683-4332-a666-082af6340779 at mountpoint /dev/vdb on host None. >2018-08-24 06:56:04.594 257256 INFO cinder.volume.manager [req-bfc476be-e405-401d-8320-7f19bb76e0b7 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Attach volume completed successfully. >2018-08-24 06:56:06.979 257256 INFO cinder.volume.manager [req-2f5500ce-3803-4483-9099-274b6d250ec7 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:56:09.104 257256 INFO cinder.volume.manager [req-57a12913-4c44-47cf-8b2e-5d8e35e430aa aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:56:11.463 257256 INFO cinder.volume.manager [req-0f77fcf7-1c29-4e73-a502-dba785a418f3 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:56:13.770 257256 INFO cinder.volume.manager [req-515ad434-e37d-4c54-a5a2-34ea1d97d92f aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:56:15.957 257256 INFO cinder.volume.manager [req-5f4e87c4-9ab2-4ae4-8509-abb7789b08f8 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:56:26.696 257256 INFO cinder.volume.manager [req-463e799f-6ed3-487e-b6df-661f2d3e4ce5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Terminate volume connection completed successfully. >2018-08-24 06:56:26.855 257256 INFO cinder.volume.manager [req-787352ba-5bc8-4b17-9852-8571dfbb3e66 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Detaching volume a5f1900b-2305-4f02-aea7-192800a80c67 from instance b6ad3537-2683-4332-a666-082af6340779. >2018-08-24 06:56:26.940 257256 INFO cinder.volume.manager [req-787352ba-5bc8-4b17-9852-8571dfbb3e66 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Detach volume completed successfully. >2018-08-24 06:56:29.457 257256 INFO cinder.volume.flows.manager.create_volume [req-b6d2861f-34c0-4cd5-8d5a-599605b66be5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Volume 3831b841-4b3a-4624-8308-c9b9703a6a2f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3831b841-4b3a-4624-8308-c9b9703a6a2f'} >2018-08-24 06:56:29.989 257256 INFO cinder.volume.flows.manager.create_volume [req-b6d2861f-34c0-4cd5-8d5a-599605b66be5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Volume volume-3831b841-4b3a-4624-8308-c9b9703a6a2f (3831b841-4b3a-4624-8308-c9b9703a6a2f): created successfully >2018-08-24 06:56:29.992 257256 INFO cinder.volume.manager [req-b6d2861f-34c0-4cd5-8d5a-599605b66be5 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Created volume successfully. >2018-08-24 06:56:32.751 257256 INFO cinder.volume.manager [req-1d9efdaf-ecca-4497-a75a-c0fe7501175c aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Create snapshot completed successfully >2018-08-24 06:56:33.146 257256 INFO cinder.volume.flows.manager.create_volume [req-ccd4ca24-fdbc-4bee-91a0-7534c71bdb3c aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Volume f7b6f93b-3f70-43f1-a008-25578b5442b7: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': 'ec9e0563-6ed8-4d4d-bac7-b6cca115b5f1', 'volume_name': 'volume-f7b6f93b-3f70-43f1-a008-25578b5442b7'} >2018-08-24 06:57:29.962 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-ccd4ca24-fdbc-4bee-91a0-7534c71bdb3c aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Unable to get remote copy information for volume volume-f7b6f93b-3f70-43f1-a008-25578b5442b7 >2018-08-24 06:57:30.266 257256 INFO cinder.volume.flows.manager.create_volume [req-ccd4ca24-fdbc-4bee-91a0-7534c71bdb3c aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Volume volume-f7b6f93b-3f70-43f1-a008-25578b5442b7 (f7b6f93b-3f70-43f1-a008-25578b5442b7): created successfully >2018-08-24 06:57:30.269 257256 INFO cinder.volume.manager [req-ccd4ca24-fdbc-4bee-91a0-7534c71bdb3c aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Created volume successfully. >2018-08-24 06:57:31.884 257256 INFO cinder.volume.manager [req-636c665d-bf4a-4089-bd4f-176ebc494097 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Deleted volume successfully. >2018-08-24 06:57:34.876 257256 INFO cinder.volume.manager [req-ceaacb4f-74e8-4a93-a056-a3ed3faa6ee1 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:57:35.107 257256 INFO cinder.volume.manager [req-1af89c69-b78f-4c53-9102-5a1460281ad4 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Delete snapshot completed successfully. >2018-08-24 06:57:37.093 257256 INFO cinder.volume.manager [req-b142acec-8c48-49dd-96f0-739a10403540 aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Deleted volume successfully. >2018-08-24 06:57:37.099 257256 WARNING cinder.db.sqlalchemy.api [req-cfeaa19b-964c-4947-8d84-174b5eef775a aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-24 06:57:37.356 257256 INFO cinder.volume.manager [req-cfeaa19b-964c-4947-8d84-174b5eef775a aedbade143024a0fa2e9cafcc3049bf1 48118a1ba4a0469aa4d7bfcd9156c67a - default default] Deleted volume successfully. >2018-08-24 06:57:42.230 257256 INFO cinder.volume.flows.manager.create_volume [req-bf1036d7-4da8-4177-9e1d-e503d216fe29 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Volume ac0d81fe-1aa0-4fe4-aa3c-f5a6f7b73f30: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ac0d81fe-1aa0-4fe4-aa3c-f5a6f7b73f30'} >2018-08-24 06:57:42.753 257256 INFO cinder.volume.flows.manager.create_volume [req-bf1036d7-4da8-4177-9e1d-e503d216fe29 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Volume volume-ac0d81fe-1aa0-4fe4-aa3c-f5a6f7b73f30 (ac0d81fe-1aa0-4fe4-aa3c-f5a6f7b73f30): created successfully >2018-08-24 06:57:42.757 257256 INFO cinder.volume.manager [req-bf1036d7-4da8-4177-9e1d-e503d216fe29 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Created volume successfully. >2018-08-24 06:57:45.737 257256 INFO cinder.volume.manager [req-3cb7f5d7-731d-4dea-a0d0-08520bc9b988 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Create snapshot completed successfully >2018-08-24 06:57:48.174 257256 INFO cinder.volume.manager [req-7b138a45-a783-4eb2-a8c1-fb9e768459a8 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Create snapshot completed successfully >2018-08-24 06:57:50.659 257256 INFO cinder.volume.manager [req-22fff869-3cd3-42d3-8ac9-195f674116d5 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Create snapshot completed successfully >2018-08-24 06:58:00.077 257256 INFO cinder.volume.manager [req-744d4a37-09d0-4e02-9dd1-d4dff0325fc6 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Delete snapshot completed successfully. >2018-08-24 06:58:00.362 257256 INFO cinder.volume.manager [req-09408a3c-ef18-4b55-9af6-ea0f90a022a1 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Delete snapshot completed successfully. >2018-08-24 06:58:00.599 257256 INFO cinder.volume.manager [req-bfb7cc1d-6520-49fd-a774-11792a812a6e 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Delete snapshot completed successfully. >2018-08-24 06:58:02.752 257256 INFO cinder.volume.manager [req-b88c9c2e-a0ac-4a82-8e3b-526230a4e1c1 0275b48c8d864f59bac5f84e481c29ad 14c343504f254cdb9a5873a13514ae7e - default default] Deleted volume successfully. >2018-08-24 06:58:08.201 257256 INFO cinder.volume.flows.manager.create_volume [req-2ff89e02-df17-4fd6-ba7b-6932cd7a4975 19dea63a5aae4835b921a31ec798e1b6 e92292c9ad9a48d1aef2e6de9ede9a24 - default default] Volume 9d481af2-95cd-4562-b09c-b8db841fd1d0: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-9d481af2-95cd-4562-b09c-b8db841fd1d0'} >2018-08-24 06:58:08.722 257256 INFO cinder.volume.flows.manager.create_volume [req-2ff89e02-df17-4fd6-ba7b-6932cd7a4975 19dea63a5aae4835b921a31ec798e1b6 e92292c9ad9a48d1aef2e6de9ede9a24 - default default] Volume volume-9d481af2-95cd-4562-b09c-b8db841fd1d0 (9d481af2-95cd-4562-b09c-b8db841fd1d0): created successfully >2018-08-24 06:58:08.726 257256 INFO cinder.volume.manager [req-2ff89e02-df17-4fd6-ba7b-6932cd7a4975 19dea63a5aae4835b921a31ec798e1b6 e92292c9ad9a48d1aef2e6de9ede9a24 - default default] Created volume successfully. >2018-08-24 06:58:11.629 257256 INFO cinder.volume.manager [req-e24bbdc6-6047-407f-9236-1ca66a6243de 19dea63a5aae4835b921a31ec798e1b6 e92292c9ad9a48d1aef2e6de9ede9a24 - default default] Create snapshot completed successfully >2018-08-24 06:58:14.654 257256 INFO cinder.volume.manager [req-821615e5-4a2b-4554-a5e7-86e15ed061d9 19dea63a5aae4835b921a31ec798e1b6 e92292c9ad9a48d1aef2e6de9ede9a24 - default default] Delete snapshot completed successfully. >2018-08-24 06:58:16.091 257256 INFO cinder.volume.manager [req-96858922-1417-4bc9-9c4e-690b718a83b2 19dea63a5aae4835b921a31ec798e1b6 e92292c9ad9a48d1aef2e6de9ede9a24 - default default] Deleted volume successfully. >2018-08-24 06:58:38.164 257256 INFO cinder.volume.flows.manager.create_volume [req-6145d66e-d03e-4580-93c1-499a83927775 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume 902ae62a-04e0-4e5d-ae09-ce8e7256a123: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-902ae62a-04e0-4e5d-ae09-ce8e7256a123', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab547cd1d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-24 06:58:38.989 257256 INFO cinder.image.image_utils [req-6145d66e-d03e-4580-93c1-499a83927775 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-24 06:58:47.902 257256 INFO os_brick.initiator.connectors.fibre_channel [-] Fibre Channel volume device not yet found. Will rescan & retry. Try number: 0. >2018-08-24 06:58:52.608 257256 INFO cinder.image.image_utils [req-6145d66e-d03e-4580-93c1-499a83927775 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-24 06:58:54.497 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-6145d66e-d03e-4580-93c1-499a83927775 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Need to remove FC Zone, building initiator target map. >2018-08-24 06:58:54.881 257256 INFO cinder.volume.flows.manager.create_volume [req-6145d66e-d03e-4580-93c1-499a83927775 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume volume-902ae62a-04e0-4e5d-ae09-ce8e7256a123 (902ae62a-04e0-4e5d-ae09-ce8e7256a123): created successfully >2018-08-24 06:58:54.884 257256 INFO cinder.volume.manager [req-6145d66e-d03e-4580-93c1-499a83927775 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Created volume successfully. >2018-08-24 06:58:55.684 257256 INFO cinder.volume.flows.manager.create_volume [req-5d0135e3-d6d0-445d-bffa-17bfcfeac7d2 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume 3a7ce2a7-0dae-4e0d-8e8f-35d31f2ce31d: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3a7ce2a7-0dae-4e0d-8e8f-35d31f2ce31d', 'source_volid': '902ae62a-04e0-4e5d-ae09-ce8e7256a123', 'source_volstatus': u'available'} >2018-08-24 06:58:57.547 257256 INFO cinder.volume.flows.manager.create_volume [req-5d0135e3-d6d0-445d-bffa-17bfcfeac7d2 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume volume-3a7ce2a7-0dae-4e0d-8e8f-35d31f2ce31d (3a7ce2a7-0dae-4e0d-8e8f-35d31f2ce31d): created successfully >2018-08-24 06:58:57.551 257256 INFO cinder.volume.manager [req-5d0135e3-d6d0-445d-bffa-17bfcfeac7d2 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Created volume successfully. >2018-08-24 06:58:58.415 257256 INFO cinder.volume.flows.manager.create_volume [req-124905e0-6cf5-467a-be44-7355bbae211b 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume 6e965029-791f-48c9-8539-cdbb6d59f2ba: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6e965029-791f-48c9-8539-cdbb6d59f2ba'} >2018-08-24 06:58:58.943 257256 INFO cinder.volume.flows.manager.create_volume [req-124905e0-6cf5-467a-be44-7355bbae211b 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume volume-6e965029-791f-48c9-8539-cdbb6d59f2ba (6e965029-791f-48c9-8539-cdbb6d59f2ba): created successfully >2018-08-24 06:58:58.947 257256 INFO cinder.volume.manager [req-124905e0-6cf5-467a-be44-7355bbae211b 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Created volume successfully. >2018-08-24 06:58:59.798 257256 INFO cinder.volume.flows.manager.create_volume [req-bd956f66-5fdd-4ebe-9213-30927c51727f 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume d0bf2843-f700-478c-bad0-46663d58ec27: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-d0bf2843-f700-478c-bad0-46663d58ec27', 'source_volid': '6e965029-791f-48c9-8539-cdbb6d59f2ba', 'source_volstatus': u'available'} >2018-08-24 06:59:31.655 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-bd956f66-5fdd-4ebe-9213-30927c51727f 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Unable to get remote copy information for volume volume-d0bf2843-f700-478c-bad0-46663d58ec27 >2018-08-24 06:59:31.920 257256 INFO cinder.volume.flows.manager.create_volume [req-bd956f66-5fdd-4ebe-9213-30927c51727f 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Volume volume-d0bf2843-f700-478c-bad0-46663d58ec27 (d0bf2843-f700-478c-bad0-46663d58ec27): created successfully >2018-08-24 06:59:31.923 257256 INFO cinder.volume.manager [req-bd956f66-5fdd-4ebe-9213-30927c51727f 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Created volume successfully. >2018-08-24 06:59:39.266 257256 INFO cinder.volume.manager [req-03674a1c-0da1-46a6-942d-f40fd043c697 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Deleted volume successfully. >2018-08-24 06:59:39.285 257256 INFO cinder.volume.manager [req-24e37942-27a8-4ee3-acc6-3943b2bceafe 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Deleted volume successfully. >2018-08-24 06:59:39.326 257256 INFO cinder.volume.manager [req-68a5c75c-b921-4bb5-934c-a045c15acdc9 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Deleted volume successfully. >2018-08-24 06:59:39.519 257256 INFO cinder.volume.manager [req-c46dc125-eda3-4ded-9860-8bf7ed8c6816 15cc3b35f7de447da8510c0592121011 24957c4a00c24c8dad3b9fe4fedc384f - default default] Deleted volume successfully. >2018-08-24 06:59:44.211 257256 INFO cinder.volume.flows.manager.create_volume [req-33e87ab6-7566-440a-84de-7d40441eab1f 2efd938288494ebbb10d1f3b591697b6 305962a5f72144079667712afa5f7356 - default default] Volume e9d76281-ecab-49c5-b62b-a8c387160f5f: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-e9d76281-ecab-49c5-b62b-a8c387160f5f'} >2018-08-24 06:59:44.823 257256 INFO cinder.volume.flows.manager.create_volume [req-33e87ab6-7566-440a-84de-7d40441eab1f 2efd938288494ebbb10d1f3b591697b6 305962a5f72144079667712afa5f7356 - default default] Volume volume-e9d76281-ecab-49c5-b62b-a8c387160f5f (e9d76281-ecab-49c5-b62b-a8c387160f5f): created successfully >2018-08-24 06:59:44.827 257256 INFO cinder.volume.manager [req-33e87ab6-7566-440a-84de-7d40441eab1f 2efd938288494ebbb10d1f3b591697b6 305962a5f72144079667712afa5f7356 - default default] Created volume successfully. >2018-08-24 06:59:47.391 257256 INFO cinder.volume.manager [req-c03e80e3-f7d7-43a6-be42-d861d546b340 2efd938288494ebbb10d1f3b591697b6 305962a5f72144079667712afa5f7356 - default default] Deleted volume successfully. >2018-08-24 07:00:10.928 257256 INFO cinder.volume.flows.manager.create_volume [req-47409420-6025-4327-8410-cc004be068fc 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] Volume 198a8f5d-3d32-4dbe-855d-a37f3bf3b4f4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-198a8f5d-3d32-4dbe-855d-a37f3bf3b4f4'} >2018-08-24 07:00:11.458 257256 INFO cinder.volume.flows.manager.create_volume [req-47409420-6025-4327-8410-cc004be068fc 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] Volume volume-198a8f5d-3d32-4dbe-855d-a37f3bf3b4f4 (198a8f5d-3d32-4dbe-855d-a37f3bf3b4f4): created successfully >2018-08-24 07:00:11.462 257256 INFO cinder.volume.manager [req-47409420-6025-4327-8410-cc004be068fc 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] Created volume successfully. >2018-08-24 07:00:14.106 257256 INFO cinder.volume.manager [req-816dda7a-7278-4807-b30d-da9c8c3ae26d 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] Deleted volume successfully. >2018-08-24 07:00:14.653 257256 INFO cinder.volume.flows.manager.create_volume [req-829526d2-5690-42d3-b044-123cc06f203a 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] Volume e74bc6a1-e6d2-4ccc-9cc2-eae51c602bae: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e74bc6a1-e6d2-4ccc-9cc2-eae51c602bae'} >2018-08-24 07:00:15.189 257256 INFO cinder.volume.flows.manager.create_volume [req-829526d2-5690-42d3-b044-123cc06f203a 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] Volume volume-e74bc6a1-e6d2-4ccc-9cc2-eae51c602bae (e74bc6a1-e6d2-4ccc-9cc2-eae51c602bae): created successfully >2018-08-24 07:00:15.193 257256 INFO cinder.volume.manager [req-829526d2-5690-42d3-b044-123cc06f203a 47c0765f9f8b471a9764939a28c87e4b 8571daf867794149b941c038a543c111 - default default] Created volume successfully. >2018-08-24 07:00:16.598 257256 INFO cinder.volume.manager [req-3b22c41b-ac6f-47f3-9077-b4a35db563c8 af1cb58ba11c4877b299d65f14bb6a9d 8cf86b6ee145443fb6a16a3bb9725cad - default default] Transfer volume completed successfully. >2018-08-24 07:00:18.582 257256 INFO cinder.volume.manager [req-d507a1e2-4fcc-4ce9-a854-dab7833523f2 32ea2cfe16734cdab166e83f26bf9db4 59fc7d7ea234447cb90d88418cf9a95d - default default] Deleted volume successfully. >2018-08-24 07:00:29.144 257256 INFO cinder.volume.flows.manager.create_volume [req-4d1f084e-f8c4-4741-a70c-3156dbd8a876 d3f922d35a4f40a3b28ca248efc85172 13198b70afe64824bef82b1e815998c1 - default default] Volume 30c9f022-af3d-4d55-9094-976d15418e3f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-30c9f022-af3d-4d55-9094-976d15418e3f'} >2018-08-24 07:00:29.667 257256 INFO cinder.volume.flows.manager.create_volume [req-4d1f084e-f8c4-4741-a70c-3156dbd8a876 d3f922d35a4f40a3b28ca248efc85172 13198b70afe64824bef82b1e815998c1 - default default] Volume volume-30c9f022-af3d-4d55-9094-976d15418e3f (30c9f022-af3d-4d55-9094-976d15418e3f): created successfully >2018-08-24 07:00:29.671 257256 INFO cinder.volume.manager [req-4d1f084e-f8c4-4741-a70c-3156dbd8a876 d3f922d35a4f40a3b28ca248efc85172 13198b70afe64824bef82b1e815998c1 - default default] Created volume successfully. >2018-08-24 07:00:33.960 257256 INFO cinder.volume.manager [req-22fe0ee3-5585-4f1c-8976-d05477a92aed d3f922d35a4f40a3b28ca248efc85172 13198b70afe64824bef82b1e815998c1 - default default] Deleted volume successfully. >2018-08-24 07:00:41.335 257256 INFO cinder.volume.flows.manager.create_volume [req-33abe070-5d0e-49c2-b445-b3b4e224b638 af33d651372845cb93fdb3498bc58525 61c1cdd4edc644338bdb983f23416d08 - default default] Volume bb9c5930-2b9b-4c90-8e83-171f0a62d447: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bb9c5930-2b9b-4c90-8e83-171f0a62d447'} >2018-08-24 07:00:42.427 257256 INFO cinder.volume.flows.manager.create_volume [req-33abe070-5d0e-49c2-b445-b3b4e224b638 af33d651372845cb93fdb3498bc58525 61c1cdd4edc644338bdb983f23416d08 - default default] Volume volume-bb9c5930-2b9b-4c90-8e83-171f0a62d447 (bb9c5930-2b9b-4c90-8e83-171f0a62d447): created successfully >2018-08-24 07:00:42.431 257256 INFO cinder.volume.manager [req-33abe070-5d0e-49c2-b445-b3b4e224b638 af33d651372845cb93fdb3498bc58525 61c1cdd4edc644338bdb983f23416d08 - default default] Created volume successfully. >2018-08-24 07:00:45.903 257256 INFO cinder.volume.manager [req-48c31a95-dd12-4a47-b313-234bdf5cd258 af33d651372845cb93fdb3498bc58525 61c1cdd4edc644338bdb983f23416d08 - default default] Create snapshot completed successfully >2018-08-24 07:00:51.259 257256 INFO cinder.volume.manager [req-e97dec5f-8912-43e8-a583-cdf193095a65 af33d651372845cb93fdb3498bc58525 61c1cdd4edc644338bdb983f23416d08 - default default] Delete snapshot completed successfully. >2018-08-24 07:00:53.215 257256 INFO cinder.volume.manager [req-dd2f17c5-1228-4f37-b29d-068cd41f4c8a af33d651372845cb93fdb3498bc58525 61c1cdd4edc644338bdb983f23416d08 - default default] Deleted volume successfully. >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 04:09:52.530 257031 ERROR oslo_service.periodic_task >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 04:09:52.533 257256 ERROR oslo_service.periodic_task >2018-08-26 04:10:54.309 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:10:54.404 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:11:32.896 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:11:32.956 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:11:39.285 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:11:39.344 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:12:32.899 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:12:32.966 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:12:39.308 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:12:39.383 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:13:39.286 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:13:39.344 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:14:39.286 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:14:39.344 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 04:39:52.357 257031 ERROR oslo_service.periodic_task >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 04:39:52.364 257256 ERROR oslo_service.periodic_task >2018-08-26 04:40:54.341 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:40:54.399 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:41:02.889 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 04:41:02.890 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 04:41:02.890 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 04:41:02.891 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 04:41:02.891 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task command) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 04:41:02.892 257031 ERROR oslo_service.periodic_task >2018-08-26 04:41:32.904 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:41:32.971 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:41:39.314 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:41:39.393 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:42:39.318 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:42:39.390 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:43:39.310 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:43:39.370 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 04:44:39.313 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 04:44:39.388 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 05:09:59.505 257256 ERROR oslo_service.periodic_task >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 05:09:59.510 257031 ERROR oslo_service.periodic_task >2018-08-26 05:11:02.909 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 05:11:02.911 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:02.912 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:02.912 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:02.912 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task command) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:02.913 257031 ERROR oslo_service.periodic_task >2018-08-26 05:11:09.317 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 05:11:09.318 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:09.319 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:09.319 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:09.319 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task command) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 05:11:09.320 257256 ERROR oslo_service.periodic_task >2018-08-26 05:11:32.921 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:11:32.981 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:11:39.328 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:11:39.385 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:12:39.335 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:12:39.416 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:13:39.341 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:13:39.425 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:14:39.326 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:14:39.385 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 05:39:54.532 257256 ERROR oslo_service.periodic_task >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 05:39:54.530 257031 ERROR oslo_service.periodic_task >2018-08-26 05:40:54.357 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:40:54.453 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:41:02.923 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 05:41:02.925 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 05:41:02.925 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 05:41:02.926 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 05:41:02.926 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task command) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 05:41:02.927 257031 ERROR oslo_service.periodic_task >2018-08-26 05:41:32.930 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:41:33.002 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:41:39.328 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:41:39.385 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:42:39.340 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:42:39.409 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 05:43:39.329 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 05:43:39.404 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 06:09:56.640 257256 ERROR oslo_service.periodic_task >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 06:09:56.642 257031 ERROR oslo_service.periodic_task >2018-08-26 06:11:02.941 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 06:11:02.942 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:02.943 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:02.943 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:02.943 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task command) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:02.944 257031 ERROR oslo_service.periodic_task >2018-08-26 06:11:09.322 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 06:11:09.323 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:09.323 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:09.323 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:09.323 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task command) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 06:11:09.324 257256 ERROR oslo_service.periodic_task >2018-08-26 06:11:39.334 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:11:39.391 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:12:32.956 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:12:33.022 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:12:39.338 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:12:39.411 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:13:39.331 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:13:39.398 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 06:39:56.384 257031 ERROR oslo_service.periodic_task >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 06:39:56.385 257256 ERROR oslo_service.periodic_task >2018-08-26 06:40:42.368 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:40:43.209 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:40:48.038 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:40:48.127 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:41:39.361 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:41:39.420 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 06:42:39.365 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 06:42:39.437 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 07:09:56.611 257031 ERROR oslo_service.periodic_task >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 07:09:56.614 257256 ERROR oslo_service.periodic_task >2018-08-26 07:11:03.022 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 07:11:03.025 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:03.025 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:03.026 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:03.026 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task command) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:03.026 257031 ERROR oslo_service.periodic_task >2018-08-26 07:11:09.362 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 07:11:09.363 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:09.363 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:09.363 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:09.364 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task command) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 07:11:09.364 257256 ERROR oslo_service.periodic_task >2018-08-26 07:11:33.035 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 07:11:33.096 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 07:11:39.370 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 07:11:39.464 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 07:12:39.380 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 07:12:39.448 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 07:39:57.447 257031 ERROR oslo_service.periodic_task >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 07:39:57.449 257256 ERROR oslo_service.periodic_task >2018-08-26 07:41:03.081 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 07:41:03.082 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:03.083 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:03.083 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:03.083 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task command) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:03.084 257031 ERROR oslo_service.periodic_task >2018-08-26 07:41:09.409 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 07:41:09.410 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:09.411 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:09.411 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:09.411 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task command) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 07:41:09.412 257256 ERROR oslo_service.periodic_task >2018-08-26 07:41:33.086 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 07:41:33.146 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 07:41:39.420 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 07:41:39.478 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 08:10:06.122 257256 ERROR oslo_service.periodic_task >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 08:10:06.121 257031 ERROR oslo_service.periodic_task >2018-08-26 08:11:03.085 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 08:11:03.087 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:03.088 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:03.088 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:03.088 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task command) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:03.089 257031 ERROR oslo_service.periodic_task >2018-08-26 08:11:09.442 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 08:11:09.442 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:09.443 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:09.443 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:09.443 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task command) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 08:11:09.444 257256 ERROR oslo_service.periodic_task >2018-08-26 08:11:33.090 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 08:11:33.149 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 08:11:39.466 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 08:11:39.537 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 08:39:59.953 257256 ERROR oslo_service.periodic_task >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 08:39:59.951 257031 ERROR oslo_service.periodic_task >2018-08-26 08:41:03.125 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 08:41:03.128 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:03.128 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:03.129 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:03.129 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task command) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:03.129 257031 ERROR oslo_service.periodic_task >2018-08-26 08:41:09.472 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 08:41:09.473 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:09.474 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:09.474 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:09.474 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task command) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 08:41:09.475 257256 ERROR oslo_service.periodic_task >2018-08-26 08:41:33.131 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 08:41:33.189 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 08:41:39.486 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 08:41:39.559 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 09:10:00.309 257256 ERROR oslo_service.periodic_task >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 09:10:00.307 257031 ERROR oslo_service.periodic_task >2018-08-26 09:11:03.151 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 09:11:03.153 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:03.154 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:03.154 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:03.154 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task command) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:03.154 257031 ERROR oslo_service.periodic_task >2018-08-26 09:11:09.507 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 09:11:09.509 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:09.509 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:09.510 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:09.510 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task command) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 09:11:09.510 257256 ERROR oslo_service.periodic_task >2018-08-26 09:11:39.532 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 09:11:39.613 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 09:12:33.167 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 09:12:33.230 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 09:40:01.869 257256 ERROR oslo_service.periodic_task >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 09:40:01.873 257031 ERROR oslo_service.periodic_task >2018-08-26 09:41:03.152 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 09:41:03.154 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:03.155 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:03.155 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:03.155 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task command) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:03.156 257031 ERROR oslo_service.periodic_task >2018-08-26 09:41:09.529 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 09:41:09.530 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:09.530 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:09.531 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:09.531 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task command) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 09:41:09.531 257256 ERROR oslo_service.periodic_task >2018-08-26 09:41:33.165 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 09:41:33.223 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 09:41:39.539 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 09:41:39.599 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 10:10:01.215 257256 ERROR oslo_service.periodic_task >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 10:10:01.213 257031 ERROR oslo_service.periodic_task >2018-08-26 10:11:09.563 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 10:11:09.566 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 10:11:09.566 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 10:11:09.567 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 10:11:09.567 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task command) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 10:11:09.567 257256 ERROR oslo_service.periodic_task >2018-08-26 10:11:33.178 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 10:11:33.236 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 10:11:39.577 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 10:11:39.637 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 10:40:03.082 257031 ERROR oslo_service.periodic_task >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 10:40:03.084 257256 ERROR oslo_service.periodic_task >2018-08-26 10:41:03.184 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 10:41:03.185 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:03.186 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:03.186 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:03.186 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task command) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:03.186 257031 ERROR oslo_service.periodic_task >2018-08-26 10:41:09.605 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 10:41:09.606 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:09.607 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:09.607 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:09.608 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task command) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 10:41:09.608 257256 ERROR oslo_service.periodic_task >2018-08-26 10:41:33.195 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 10:41:33.253 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 10:41:39.615 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 10:41:39.673 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 11:10:06.970 257256 ERROR oslo_service.periodic_task >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 11:10:06.969 257031 ERROR oslo_service.periodic_task >2018-08-26 11:11:03.221 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 11:11:03.223 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:03.223 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:03.224 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:03.224 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task command) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:03.224 257031 ERROR oslo_service.periodic_task >2018-08-26 11:11:09.616 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 11:11:09.618 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:09.619 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:09.619 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:09.620 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task command) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 11:11:09.620 257256 ERROR oslo_service.periodic_task >2018-08-26 11:11:33.244 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 11:11:33.326 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 11:11:39.627 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 11:11:39.685 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 11:40:03.581 257031 ERROR oslo_service.periodic_task >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 11:40:03.583 257256 ERROR oslo_service.periodic_task >2018-08-26 11:41:03.222 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 11:41:03.224 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:03.225 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:03.225 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:03.225 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task command) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:03.225 257031 ERROR oslo_service.periodic_task >2018-08-26 11:41:09.610 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 11:41:09.611 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:09.612 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:09.612 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:09.612 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task command) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 11:41:09.613 257256 ERROR oslo_service.periodic_task >2018-08-26 11:41:33.251 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 11:41:33.325 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 11:41:39.622 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 11:41:39.680 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 12:10:09.619 257256 ERROR oslo_service.periodic_task >2018-08-26 12:11:03.254 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 12:11:03.257 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:03.257 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:03.257 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:03.258 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task command) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:03.258 257031 ERROR oslo_service.periodic_task >2018-08-26 12:11:09.630 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 12:11:09.631 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:09.631 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:09.632 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:09.632 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task command) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 12:11:09.632 257256 ERROR oslo_service.periodic_task >2018-08-26 12:11:33.272 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 12:11:33.331 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 12:11:39.640 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 12:11:39.699 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 12:40:05.658 257256 ERROR oslo_service.periodic_task >2018-08-26 12:40:54.720 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 12:40:54.809 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 12:41:03.256 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 12:41:03.256 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 12:41:03.257 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 12:41:03.257 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 12:41:03.258 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task command) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 12:41:03.258 257031 ERROR oslo_service.periodic_task >2018-08-26 12:41:33.267 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 12:41:33.329 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 13:10:05.776 257256 ERROR oslo_service.periodic_task >2018-08-26 13:11:03.302 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 13:11:03.303 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:03.304 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:03.304 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:03.304 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task command) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:03.305 257031 ERROR oslo_service.periodic_task >2018-08-26 13:11:09.707 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 13:11:09.707 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:09.708 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:09.708 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:09.708 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task command) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 13:11:09.709 257256 ERROR oslo_service.periodic_task >2018-08-26 13:11:33.314 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 13:11:33.372 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 13:11:39.719 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 13:11:39.776 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 13:40:09.430 257256 ERROR oslo_service.periodic_task >2018-08-26 13:41:03.313 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 13:41:03.315 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:03.315 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:03.316 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:03.316 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task command) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:03.316 257031 ERROR oslo_service.periodic_task >2018-08-26 13:41:09.721 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 13:41:09.722 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:09.722 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:09.723 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:09.723 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task command) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 13:41:09.723 257256 ERROR oslo_service.periodic_task >2018-08-26 13:41:33.338 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 13:41:33.412 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 13:41:39.731 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 13:41:39.788 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 14:10:25.272 257256 ERROR oslo_service.periodic_task >2018-08-26 14:10:33.343 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 14:10:33.343 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 14:10:33.344 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 14:10:33.344 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 14:10:33.344 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task command) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 14:10:33.345 257031 ERROR oslo_service.periodic_task >2018-08-26 14:11:09.733 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 14:11:09.736 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 14:11:09.736 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 14:11:09.736 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 14:11:09.737 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task command) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 14:11:09.737 257256 ERROR oslo_service.periodic_task >2018-08-26 14:11:33.354 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 14:11:33.420 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 14:11:39.749 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 14:11:39.810 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 14:40:12.171 257256 ERROR oslo_service.periodic_task >2018-08-26 14:41:03.358 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 14:41:03.361 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:03.361 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:03.361 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:03.362 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task command) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:03.362 257031 ERROR oslo_service.periodic_task >2018-08-26 14:41:09.723 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 14:41:09.724 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:09.725 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:09.725 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:09.725 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task command) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 14:41:09.726 257256 ERROR oslo_service.periodic_task >2018-08-26 14:41:33.371 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 14:41:33.431 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 14:41:39.748 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 14:41:39.811 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 15:10:33.395 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:33.397 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:33.397 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:33.397 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:33.398 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task command) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:33.398 257031 ERROR oslo_service.periodic_task >2018-08-26 15:10:39.766 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:39.767 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:39.767 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:39.767 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:39.768 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task command) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 15:10:39.768 257256 ERROR oslo_service.periodic_task >2018-08-26 15:11:33.406 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 15:11:33.463 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 15:11:39.779 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 15:11:39.854 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 15:41:03.429 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 15:41:03.431 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:03.431 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:03.432 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:03.432 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task command) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:03.432 257031 ERROR oslo_service.periodic_task >2018-08-26 15:41:09.767 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 15:41:09.768 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:09.769 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:09.769 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:09.769 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task command) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 15:41:09.770 257256 ERROR oslo_service.periodic_task >2018-08-26 15:41:33.452 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 15:41:33.555 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 15:41:39.779 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 15:41:39.837 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 16:12:19.465 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 99.67 sec >2018-08-26 16:12:19.613 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 46.18 sec >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 16:13:57.953 257031 ERROR oslo_service.periodic_task >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 16:13:57.954 257256 ERROR oslo_service.periodic_task >2018-08-26 16:14:49.486 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 16:14:49.489 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.489 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.489 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.490 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task command) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.490 257256 ERROR oslo_service.periodic_task >2018-08-26 16:14:49.613 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 16:14:49.614 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.614 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.615 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.615 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task command) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 16:14:49.615 257031 ERROR oslo_service.periodic_task >2018-08-26 16:15:19.497 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 16:15:19.567 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 16:15:19.626 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 16:15:19.695 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 16:40:23.063 257256 ERROR oslo_service.periodic_task >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 16:40:23.064 257031 ERROR oslo_service.periodic_task >2018-08-26 16:41:22.492 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 16:41:22.576 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 16:41:22.640 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 16:41:22.737 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 17:12:16.563 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 57.11 sec >2018-08-26 17:12:16.586 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 56.98 sec >2018-08-26 17:15:26.562 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 17:15:26.583 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 17:18:36.561 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 17:18:36.583 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 17:21:46.559 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 17:21:46.580 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 17:24:58.592 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 72.03 sec >2018-08-26 17:24:58.613 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 72.03 sec >2018-08-26 17:28:06.557 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 67.96 sec >2018-08-26 17:28:06.579 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 67.96 sec >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 17:30:09.977 257031 ERROR oslo_service.periodic_task >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 17:30:09.975 257256 ERROR oslo_service.periodic_task >2018-08-26 17:30:09.979 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 3.40 sec >2018-08-26 17:30:09.979 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 3.42 sec >2018-08-26 17:30:09.986 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.986 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.986 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.986 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task command) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 17:30:09.987 2570312018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task command) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257031 ERROR oslo_service.periodic_task > ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:30:09.987 257256 ERROR oslo_service.periodic_task >2018-08-26 17:31:10.040 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 17:31:10.047 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 17:31:10.126 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 17:31:10.131 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 17:50:23.079 257031 ERROR oslo_service.periodic_task >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 17:50:23.079 257256 ERROR oslo_service.periodic_task >2018-08-26 17:50:23.082 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 13.10 sec >2018-08-26 17:50:23.082 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 13.09 sec >2018-08-26 17:50:23.089 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.089 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.090 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.090 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.090 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.090 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.090 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.090 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.091 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.091 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task command) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.091 257031 ERROR oslo_service.periodic_task >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task command) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 17:50:23.091 257256 ERROR oslo_service.periodic_task >2018-08-26 17:51:23.112 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 17:51:23.113 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 17:51:23.511 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 17:51:23.513 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 18:21:38.202 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 75.12 sec >2018-08-26 18:21:38.224 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 75.13 sec >2018-08-26 18:24:46.975 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 68.77 sec >2018-08-26 18:24:46.996 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 68.77 sec >2018-08-26 18:27:56.973 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:27:56.995 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:31:06.972 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:31:06.994 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:34:16.971 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 18:34:16.992 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 18:37:26.971 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 18:37:26.993 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 10.00 sec >2018-08-26 18:40:40.137 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 73.16 sec >2018-08-26 18:40:40.158 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 73.17 sec >2018-08-26 18:43:46.968 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 66.83 sec >2018-08-26 18:43:46.989 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 66.83 sec >2018-08-26 18:46:56.967 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:46:56.988 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:50:06.968 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:50:06.989 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:53:16.964 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 18:53:16.986 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 18:56:26.963 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:56:26.984 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 18:59:36.962 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 18:59:36.984 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:02:46.960 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:02:46.982 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:05:56.960 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:05:56.980 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:09:06.958 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:09:06.980 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:12:16.957 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:12:16.978 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:15:26.956 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:15:26.975 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:18:36.954 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 19:18:36.975 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:21:46.954 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:21:46.975 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:24:56.951 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 19:24:56.974 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:28:06.951 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 19:28:06.972 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 19:31:16.951 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:31:16.971 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:34:26.948 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:34:26.968 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:37:36.948 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:37:36.969 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:40:46.946 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:40:46.967 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:44:00.151 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 73.20 sec >2018-08-26 19:44:00.171 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 73.20 sec >2018-08-26 19:47:06.944 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 66.79 sec >2018-08-26 19:47:06.965 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 66.79 sec >2018-08-26 19:50:16.943 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:50:16.964 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:53:26.943 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:53:26.964 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:56:36.941 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 19:56:36.961 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 19:58:32.505 257031 ERROR oslo_service.periodic_task >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 19:58:32.506 257256 ERROR oslo_service.periodic_task >2018-08-26 19:58:36.957 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.957 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.958 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.958 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.958 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task command) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.958 257256 ERROR oslo_service.periodic_task >2018-08-26 19:58:36.977 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.978 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.978 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.978 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.979 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task command) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-26 19:58:36.979 257031 ERROR oslo_service.periodic_task >2018-08-26 19:59:36.968 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 19:59:36.987 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 19:59:37.026 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 19:59:37.046 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 20:21:25.948 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 49.00 sec >2018-08-26 20:21:25.969 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 49.02 sec >2018-08-26 20:24:35.936 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 20:24:35.955 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 20:27:45.964 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-26 20:27:45.986 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-26 20:30:55.960 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 20:30:55.981 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 20:34:05.971 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-26 20:34:05.993 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-26 20:37:15.915 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-26 20:37:15.936 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-26 20:40:25.916 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 20:40:25.936 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 20:41:45.290 257031 ERROR oslo_service.periodic_task >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 20:41:45.289 257256 ERROR oslo_service.periodic_task >2018-08-26 20:42:55.930 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 20:42:55.932 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.933 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.933 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.933 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task command) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.933 257256 ERROR oslo_service.periodic_task >2018-08-26 20:42:55.949 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 20:42:55.950 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.951 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.951 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.951 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task command) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 20:42:55.952 257031 ERROR oslo_service.periodic_task >2018-08-26 20:43:25.946 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 20:43:25.960 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 20:43:26.003 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 20:43:26.019 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 20:53:45.171 257256 ERROR oslo_service.periodic_task >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 20:53:45.172 257031 ERROR oslo_service.periodic_task >2018-08-26 20:54:55.931 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 20:54:55.933 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.934 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.934 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.934 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task command) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.934 257256 ERROR oslo_service.periodic_task >2018-08-26 20:54:55.960 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 20:54:55.960 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.961 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.961 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.961 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task command) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 20:54:55.962 257031 ERROR oslo_service.periodic_task >2018-08-26 20:55:25.947 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 20:55:25.972 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 20:55:26.005 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 20:55:26.028 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 21:25:29.217 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 63.29 sec >2018-08-26 21:25:29.238 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 63.28 sec >2018-08-26 21:28:39.197 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 21:28:39.217 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 21:31:49.197 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 21:31:49.218 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 21:34:59.198 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 21:34:59.219 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 10.02 sec >2018-08-26 21:38:09.198 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 21:38:09.221 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 21:41:19.198 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 21:41:19.220 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 21:44:29.221 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-26 21:44:29.244 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-26 21:47:39.204 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 21:47:39.221 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 21:50:50.157 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.95 sec >2018-08-26 21:50:50.181 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.96 sec >2018-08-26 21:53:59.253 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.09 sec >2018-08-26 21:53:59.274 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.09 sec >2018-08-26 21:57:09.201 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-26 21:57:09.222 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-26 22:00:19.207 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:00:19.228 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-26 22:03:29.204 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 22:03:29.225 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 22:06:39.204 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:06:39.227 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:09:49.204 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 22:09:49.226 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 22:12:59.205 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:12:59.226 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:16:09.206 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:16:09.227 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:19:19.206 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:19:19.227 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:22:29.209 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:22:29.230 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:25:39.250 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-26 22:25:39.271 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-26 22:28:49.209 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-26 22:28:49.229 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.96 sec >2018-08-26 22:32:01.365 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 72.16 sec >2018-08-26 22:32:01.386 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 72.16 sec >2018-08-26 22:35:09.210 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 67.85 sec >2018-08-26 22:35:09.231 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 67.84 sec >2018-08-26 22:38:19.768 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.56 sec >2018-08-26 22:38:19.789 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.56 sec >2018-08-26 22:41:29.212 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.44 sec >2018-08-26 22:41:29.233 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.44 sec >2018-08-26 22:44:39.211 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:44:39.234 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:47:49.214 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:47:49.236 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:50:59.216 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:50:59.237 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:54:09.214 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 22:54:09.234 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 22:57:19.217 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 22:57:19.241 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:00:29.217 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:00:29.237 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 23:02:15.709 257031 ERROR oslo_service.periodic_task >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-26 23:02:15.708 257256 ERROR oslo_service.periodic_task >2018-08-26 23:02:59.228 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 23:02:59.229 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.229 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.230 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.230 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task command) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.231 257031 ERROR oslo_service.periodic_task >2018-08-26 23:02:59.237 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-26 23:02:59.237 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.238 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.238 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.238 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task command) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-26 23:02:59.238 257256 ERROR oslo_service.periodic_task >2018-08-26 23:03:29.243 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 23:03:29.252 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-26 23:03:29.318 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 23:03:29.323 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-26 23:16:19.539 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 50.32 sec >2018-08-26 23:16:19.559 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 50.32 sec >2018-08-26 23:19:29.572 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-26 23:19:29.592 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-26 23:22:39.486 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.91 sec >2018-08-26 23:22:39.507 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.91 sec >2018-08-26 23:25:49.530 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-26 23:25:49.550 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-26 23:28:59.525 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 23:28:59.546 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:32:09.545 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-26 23:32:09.564 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-26 23:35:19.503 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.96 sec >2018-08-26 23:35:19.524 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.96 sec >2018-08-26 23:38:29.503 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:38:29.524 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:41:39.477 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-26 23:41:39.498 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-26 23:44:49.476 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:44:49.497 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 23:47:59.476 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:47:59.497 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:51:09.476 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-26 23:51:09.495 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:54:19.475 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:54:19.496 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:57:29.474 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-26 23:57:29.494 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:00:39.474 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:00:39.497 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:03:49.470 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:03:49.491 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:06:59.471 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:06:59.491 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:10:09.468 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:10:09.489 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:13:19.496 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 00:13:19.519 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 00:16:29.465 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 00:16:29.488 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 00:19:39.465 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:19:39.486 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:22:49.463 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:22:49.485 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:25:59.462 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:25:59.483 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:29:09.461 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:29:09.482 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:32:19.461 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:32:19.482 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:35:29.458 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:35:29.478 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 00:38:39.456 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:38:39.477 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 00:41:50.227 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.76 sec >2018-08-27 00:41:50.248 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.77 sec >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 00:44:05.069 257031 ERROR oslo_service.periodic_task >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 00:44:05.070 257256 ERROR oslo_service.periodic_task >2018-08-27 00:44:05.073 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 14.83 sec >2018-08-27 00:44:05.073 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 14.84 sec >2018-08-27 00:44:05.080 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.080 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.080 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.080 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.081 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.081 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.081 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.081 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.081 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.081 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task command) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 00:44:05.082 2570312018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task command) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.082 257031 ERROR oslo_service.periodic_task > ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 00:44:05.082 257256 ERROR oslo_service.periodic_task >2018-08-27 00:45:05.135 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 00:45:05.135 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 00:45:05.629 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 00:45:05.629 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 01:06:59.264 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 54.20 sec >2018-08-27 01:06:59.285 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 54.20 sec >2018-08-27 01:07:59.273 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.275 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.275 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.276 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.276 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task command) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.276 257256 ERROR oslo_service.periodic_task >2018-08-27 01:07:59.296 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.297 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.297 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.297 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.298 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task command) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 01:07:59.298 257031 ERROR oslo_service.periodic_task >2018-08-27 01:08:59.297 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 01:08:59.305 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 01:08:59.379 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 01:08:59.380 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 01:37:00.900 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 61.63 sec >2018-08-27 01:37:00.921 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 61.64 sec >2018-08-27 01:40:10.885 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 01:40:10.906 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 01:43:20.938 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 01:43:20.959 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.07 sec >2018-08-27 01:46:30.972 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 01:46:30.992 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-27 01:49:40.881 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.91 sec >2018-08-27 01:49:40.903 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.91 sec >2018-08-27 01:52:50.906 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 01:52:50.926 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 10.01 sec >2018-08-27 01:56:00.963 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 01:56:00.983 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 01:57:30.977 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 01:57:30.980 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.981 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.981 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.981 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task command) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.982 257256 ERROR oslo_service.periodic_task >2018-08-27 01:57:30.994 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 01:57:30.994 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.995 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.995 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.995 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task command) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 01:57:30.996 257031 ERROR oslo_service.periodic_task >2018-08-27 01:58:00.990 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 01:58:01.005 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 01:58:01.048 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 01:58:01.064 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 02:23:15.582 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 74.61 sec >2018-08-27 02:23:15.604 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 74.61 sec >2018-08-27 02:26:25.578 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 02:26:25.602 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:29:35.582 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:29:35.603 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:32:45.601 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 02:32:45.622 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 02:35:55.580 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 02:35:55.601 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 02:39:05.582 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:39:05.602 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:42:15.631 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-27 02:42:15.651 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 02:45:25.580 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 02:45:25.601 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 02:48:35.583 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:48:35.604 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:51:45.581 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 02:51:45.601 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 02:54:55.661 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.08 sec >2018-08-27 02:54:55.682 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.08 sec >2018-08-27 02:58:05.585 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 02:58:05.606 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 03:01:15.585 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:01:15.606 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:04:25.586 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:04:25.607 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:07:35.588 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:07:35.609 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:10:45.592 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:10:45.606 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:13:55.587 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:13:55.608 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:15:25.600 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 03:15:25.603 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.603 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.604 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.604 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task command) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.604 257256 ERROR oslo_service.periodic_task >2018-08-27 03:15:25.615 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 03:15:25.615 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.616 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.616 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.616 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task command) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 03:15:25.617 257031 ERROR oslo_service.periodic_task >2018-08-27 03:15:55.614 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 03:15:55.626 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 03:15:55.675 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 03:15:55.686 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 03:31:55.396 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 59.80 sec >2018-08-27 03:31:55.416 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 59.80 sec >2018-08-27 03:35:05.445 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 03:35:05.465 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 03:38:15.391 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 03:38:15.413 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 03:41:25.395 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:41:25.415 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 03:44:35.347 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 03:44:35.368 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 03:47:45.412 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 03:47:45.432 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 03:50:55.410 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 03:50:55.431 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 03:54:05.404 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 03:54:05.425 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 03:57:15.399 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 03:57:15.420 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 04:00:25.342 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 04:00:25.363 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 04:03:35.348 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 04:03:35.362 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 04:06:31.795 257031 ERROR oslo_service.periodic_task >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 04:06:31.794 257256 ERROR oslo_service.periodic_task >2018-08-27 04:06:31.798 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 56.43 sec >2018-08-27 04:06:31.798 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 56.45 sec >2018-08-27 04:06:31.805 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.805 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.805 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.805 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.805 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.805 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task command) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 04:06:31.806 2572562018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task command) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257256 ERROR oslo_service.periodic_task > ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 04:06:31.806 257031 ERROR oslo_service.periodic_task >2018-08-27 04:07:31.848 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 04:07:31.851 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 04:07:31.933 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 04:07:31.948 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 04:30:15.575 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 43.76 sec >2018-08-27 04:30:15.596 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 43.79 sec >2018-08-27 04:33:25.523 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 04:33:25.543 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 04:36:35.552 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 04:36:35.571 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 04:39:45.575 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 04:39:45.595 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 04:42:55.524 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 04:42:55.545 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 04:46:05.604 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.07 sec >2018-08-27 04:46:05.630 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.08 sec >2018-08-27 04:49:15.526 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 04:49:15.547 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 04:52:25.530 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 04:52:25.551 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 04:53:45.058 257031 ERROR oslo_service.periodic_task >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 04:53:45.060 257256 ERROR oslo_service.periodic_task >2018-08-27 04:54:55.540 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 04:54:55.541 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.542 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.542 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.542 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task command) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.543 257256 ERROR oslo_service.periodic_task >2018-08-27 04:54:55.558 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 04:54:55.558 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.559 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.559 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.559 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task command) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 04:54:55.560 257031 ERROR oslo_service.periodic_task >2018-08-27 04:55:25.569 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 04:55:25.589 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 04:55:25.656 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 04:55:25.658 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 05:11:55.231 257031 ERROR oslo_service.periodic_task >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 05:11:55.230 257256 ERROR oslo_service.periodic_task >2018-08-27 05:12:55.542 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 05:12:55.544 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.544 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.544 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.545 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task command) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.545 257256 ERROR oslo_service.periodic_task >2018-08-27 05:12:55.561 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 05:12:55.562 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.562 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.563 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.563 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task command) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 05:12:55.563 257031 ERROR oslo_service.periodic_task >2018-08-27 05:13:25.555 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 05:13:25.572 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 05:13:25.613 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 05:13:25.628 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 06:28:53.985 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 88.46 sec >2018-08-27 06:28:54.000 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 88.41 sec >2018-08-27 06:32:03.996 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-27 06:32:04.017 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-27 06:35:14.059 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 06:35:14.079 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 06:38:23.955 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.90 sec >2018-08-27 06:38:23.976 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.89 sec >2018-08-27 06:41:33.954 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 06:41:33.976 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 10.02 sec >2018-08-27 06:44:43.952 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 06:44:43.973 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 06:47:53.997 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-27 06:47:54.018 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 06:51:04.012 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-27 06:51:04.033 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-27 06:54:13.943 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 06:54:13.967 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 06:57:24.004 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 06:57:24.024 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 07:00:34.018 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-27 07:00:34.039 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.01 sec >2018-08-27 07:03:43.946 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 07:03:43.956 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 07:06:54.001 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 07:06:54.022 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 07:10:03.931 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 07:10:03.952 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 07:13:13.966 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-27 07:13:13.987 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 07:16:23.949 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 07:16:23.966 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 07:19:33.925 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 07:19:33.945 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 07:22:43.978 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 07:22:43.998 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 07:25:35.891 257031 ERROR oslo_service.periodic_task >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 07:25:35.890 257256 ERROR oslo_service.periodic_task >2018-08-27 07:25:35.894 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 51.90 sec >2018-08-27 07:25:35.894 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 51.92 sec >2018-08-27 07:25:35.902 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.903 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.903 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.903 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.903 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.903 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.903 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task command) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257031 ERROR oslo_service.periodic_task >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task command) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:25:35.904 257256 ERROR oslo_service.periodic_task >2018-08-27 07:26:35.940 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 07:26:35.940 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 07:26:36.382 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 07:26:36.383 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 07:40:38.159 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 62.27 sec >2018-08-27 07:40:38.166 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 62.27 sec >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 07:43:32.156 257031 ERROR oslo_service.periodic_task >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 07:43:32.154 257256 ERROR oslo_service.periodic_task >2018-08-27 07:43:32.159 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 54.00 sec >2018-08-27 07:43:32.159 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 54.00 sec >2018-08-27 07:43:32.166 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.166 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.166 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.167 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.167 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.167 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.168 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task command) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 07:43:32.167 2572562018-08-27 07:43:32.168 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 > ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.167 257256 ERROR oslo_service.periodic_task >2018-08-27 07:43:32.168 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.169 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task command) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 07:43:32.169 257031 ERROR oslo_service.periodic_task >2018-08-27 07:44:32.200 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 07:44:32.211 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 07:44:32.295 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 07:44:32.296 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 08:10:36.370 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 64.21 sec >2018-08-27 08:10:36.391 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 64.23 sec >2018-08-27 08:13:46.357 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 08:13:46.378 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 08:16:56.307 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 08:16:56.328 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 08:20:06.377 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 08:20:06.399 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.07 sec >2018-08-27 08:21:06.384 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.385 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.385 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.386 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.386 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task command) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.386 257031 ERROR oslo_service.periodic_task >2018-08-27 08:21:06.407 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.407 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.407 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.408 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.408 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task command) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 08:21:06.408 257256 ERROR oslo_service.periodic_task >2018-08-27 08:22:06.418 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 08:22:06.430 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 08:22:06.499 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 08:22:06.502 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 08:47:32.492 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 86.12 sec >2018-08-27 08:47:32.513 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 86.11 sec >2018-08-27 08:50:42.521 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 08:50:42.542 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 08:53:52.529 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 08:53:52.544 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 10.00 sec >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 08:55:13.924 257031 ERROR oslo_service.periodic_task >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 08:55:13.925 257256 ERROR oslo_service.periodic_task >2018-08-27 08:56:22.546 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 08:56:22.548 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.548 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.549 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.549 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task command) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.549 257031 ERROR oslo_service.periodic_task >2018-08-27 08:56:22.552 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 08:56:22.553 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.553 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.553 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.554 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task command) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 08:56:22.554 257256 ERROR oslo_service.periodic_task >2018-08-27 08:56:52.558 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 08:56:52.566 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 08:56:52.636 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 08:56:52.636 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 09:16:22.556 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 09:16:22.559 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.559 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.560 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.560 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task command) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.560 257031 ERROR oslo_service.periodic_task >2018-08-27 09:16:22.590 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-27 09:16:22.591 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.591 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.592 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.592 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task command) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-27 09:16:22.592 257256 ERROR oslo_service.periodic_task >2018-08-27 09:16:52.575 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 09:16:52.605 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 09:16:52.659 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 09:16:52.680 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 09:47:35.120 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 42.56 sec >2018-08-27 09:47:35.141 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 42.55 sec >2018-08-27 09:50:45.163 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-27 09:50:45.189 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 09:53:55.118 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 09:53:55.139 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 09:57:05.116 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 09:57:05.138 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:00:15.169 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 10:00:15.190 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 10:03:25.113 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 10:03:25.135 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 10:06:37.711 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 72.60 sec >2018-08-27 10:06:37.730 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 72.59 sec >2018-08-27 10:09:45.113 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 67.40 sec >2018-08-27 10:09:45.133 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 67.40 sec >2018-08-27 10:12:55.109 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:12:55.131 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:16:05.108 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:16:05.129 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:19:15.107 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:19:15.131 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:22:25.107 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:22:25.127 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:25:35.105 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:25:35.126 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:28:45.104 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:28:45.126 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:31:55.102 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:31:55.123 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:35:05.101 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:35:05.122 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:38:15.158 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 10:38:15.183 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 10:41:25.097 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 10:41:25.121 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 10:44:35.097 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:44:35.118 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:47:45.096 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:47:45.116 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:50:55.095 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 10:50:55.115 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:54:05.094 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:54:05.115 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:57:15.094 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 10:57:15.116 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:00:25.124 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.03 sec >2018-08-27 11:00:25.145 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 11:03:35.091 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 11:03:35.117 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 11:06:45.089 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:06:45.111 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 11:09:55.088 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:09:55.109 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:13:05.156 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 11:13:05.177 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 11:16:15.087 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 11:16:15.109 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 11:19:25.147 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 11:19:25.166 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.06 sec >2018-08-27 11:22:35.131 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 11:22:35.152 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 9.98 sec >2018-08-27 11:25:45.099 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 11:25:45.120 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 11:28:55.084 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 11:28:55.106 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 11:32:05.080 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 11:32:05.102 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 11:35:15.080 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 11:35:15.100 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:38:25.081 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:38:25.101 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:41:35.077 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:41:35.098 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:44:45.079 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:44:45.099 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:47:55.077 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:47:55.097 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:51:05.074 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:51:05.096 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:54:15.075 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:54:15.096 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:57:25.075 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 11:57:25.096 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 12:00:35.072 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 12:00:35.093 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 12:03:45.070 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 12:03:45.092 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 12:06:55.069 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 12:06:55.090 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 12:07:58.672 257256 ERROR oslo_service.periodic_task >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 12:07:58.671 257031 ERROR oslo_service.periodic_task >2018-08-27 12:08:58.105 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 12:08:58.116 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 12:08:59.325 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 12:08:59.325 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 12:30:55.078 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.079 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.080 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.080 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.080 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task command) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.080 257031 ERROR oslo_service.periodic_task >2018-08-27 12:30:55.096 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.097 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.097 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.097 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.097 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task command) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 12:30:55.098 257256 ERROR oslo_service.periodic_task >2018-08-27 12:31:55.099 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 12:31:55.103 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 12:31:55.189 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 12:31:55.191 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 13:02:53.051 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 58.00 sec >2018-08-27 13:02:53.071 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 57.97 sec >2018-08-27 13:06:03.014 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.96 sec >2018-08-27 13:06:03.035 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.96 sec >2018-08-27 13:09:13.015 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 13:09:13.035 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 13:12:23.015 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 13:12:23.032 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 13:15:33.009 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 13:15:33.030 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 13:18:43.074 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-27 13:18:43.095 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 10.09 sec >2018-08-27 13:21:53.009 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 13:21:53.030 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.93 sec >2018-08-27 13:25:03.008 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 13:25:03.029 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 13:28:13.055 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.04 sec >2018-08-27 13:28:13.077 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.05 sec >2018-08-27 13:31:23.007 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 13:31:23.027 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 13:34:33.057 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 13:34:33.078 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 10.07 sec >2018-08-27 13:37:43.002 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.94 sec >2018-08-27 13:37:43.027 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.95 sec >2018-08-27 13:40:53.121 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.11 sec >2018-08-27 13:40:53.142 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.11 sec >2018-08-27 13:44:03.001 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.87 sec >2018-08-27 13:44:03.021 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.87 sec >2018-08-27 13:47:13.080 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.08 sec >2018-08-27 13:47:13.102 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.08 sec >2018-08-27 13:50:23.000 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 13:50:23.021 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.92 sec >2018-08-27 13:53:32.997 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 13:53:33.019 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 13:56:42.997 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 13:56:43.017 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 13:59:52.997 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 13:59:53.018 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 14:03:02.994 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 14:03:03.016 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.99 sec >2018-08-27 14:06:13.021 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 14:06:13.038 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.02 sec >2018-08-27 14:09:22.995 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.97 sec >2018-08-27 14:09:23.015 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 69.98 sec >2018-08-27 14:12:32.993 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 14:12:33.013 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 14:15:42.992 257031 WARNING oslo.service.loopingcall [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 14:15:43.014 257256 WARNING oslo.service.loopingcall [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 70.00 sec >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 14:17:34.763 257031 ERROR oslo_service.periodic_task >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: IndexError: list index out of range >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task return self.result[key] >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task IndexError: list index out of range >2018-08-27 14:17:34.762 257256 ERROR oslo_service.periodic_task >2018-08-27 14:17:43.006 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.007 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.007 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.007 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.008 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task command) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.008 257031 ERROR oslo_service.periodic_task >2018-08-27 14:17:43.014 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.015 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.016 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.016 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.016 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task command) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-27 14:17:43.017 257256 ERROR oslo_service.periodic_task >2018-08-27 14:18:43.019 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 14:18:43.041 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-27 14:18:43.098 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-27 14:18:43.122 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 02:02:44.057 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 02:02:44.130 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 02:02:44.862 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 02:02:44.863 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 02:32:46.064 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 02:32:46.117 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 02:32:46.150 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 02:32:46.177 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 03:02:46.092 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 03:02:46.110 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 03:02:47.470 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 03:02:47.470 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 03:32:50.045 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 03:32:50.125 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 03:32:50.141 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 03:32:50.185 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 04:02:50.059 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 04:02:50.118 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 04:02:50.145 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 04:02:50.177 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 04:32:50.058 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 04:32:50.143 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 04:32:50.173 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 04:32:50.221 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 05:02:58.065 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 05:02:58.138 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 05:02:58.179 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 05:02:58.244 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 05:32:50.045 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 05:32:50.157 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 05:32:51.085 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 05:32:51.086 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 06:02:58.089 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 06:02:58.162 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 06:02:58.198 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 06:02:58.258 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 06:32:58.101 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 06:32:58.175 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 06:32:58.213 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 06:32:58.276 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 07:02:58.129 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 07:02:58.210 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 07:02:58.252 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 07:02:58.345 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 07:32:58.119 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 07:32:58.240 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 07:32:58.252 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 07:32:58.333 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 08:02:58.085 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 08:02:58.180 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 08:02:58.245 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-28 08:02:58.315 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-28 08:49:32.099 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-64008ea6-147b-41cb-8c16-200daa251b77 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Unable to get remote copy information for volume volume-27f45d41-1cc2-4a9f-8af7-49de9acada2e >2018-08-28 08:49:33.518 257031 INFO cinder.compute.nova [req-64008ea6-147b-41cb-8c16-200daa251b77 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Nova event response: {u'status': u'completed', u'tag': u'27f45d41-1cc2-4a9f-8af7-49de9acada2e', u'name': u'volume-extended', u'server_uuid': u'9cdcfd6a-d4be-4d2c-aae6-abca865d29ea', u'code': 200}. >2018-08-28 08:49:33.543 257031 INFO cinder.volume.manager [req-64008ea6-147b-41cb-8c16-200daa251b77 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Extend volume completed successfully. >2018-08-29 02:35:44.211 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error has occurred: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n': ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:44.239 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error has occurred: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n': ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.455 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.456 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.457 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.458 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] CLI Exception output: > command: ['svcinfo', 'lsmdiskgrp', '-bytes', '-delim', '!', '"NonDR_DS8KBH00_Pool"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.459 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Failed to get pool NonDR_DS8KBH00_Pool info. Exception: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsmdiskgrp', '-bytes', '-delim', '!', '"NonDR_DS8KBH00_Pool"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >..: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsmdiskgrp', '-bytes', '-delim', '!', '"NonDR_DS8KBH00_Pool"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: KeyError: 'pool_name' >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2415, in _report_driver_status >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task self._append_volume_stats(volume_stats) >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2429, in _append_volume_stats >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task pool_name = pool['pool_name'] >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task KeyError: 'pool_name' >2018-08-29 02:35:47.459 257031 ERROR oslo_service.periodic_task >2018-08-29 02:35:47.481 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.482 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.483 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.484 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] CLI Exception output: > command: ['svcinfo', 'lsmdiskgrp', '-bytes', '-delim', '!', '"NonDR_DS8KBH00_Pool"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 02:35:47.484 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Failed to get pool NonDR_DS8KBH00_Pool info. Exception: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsmdiskgrp', '-bytes', '-delim', '!', '"NonDR_DS8KBH00_Pool"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >..: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsmdiskgrp', '-bytes', '-delim', '!', '"NonDR_DS8KBH00_Pool"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: KeyError: 'pool_name' >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2415, in _report_driver_status >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task self._append_volume_stats(volume_stats) >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2429, in _append_volume_stats >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task pool_name = pool['pool_name'] >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task KeyError: 'pool_name' >2018-08-29 02:35:47.485 257256 ERROR oslo_service.periodic_task >2018-08-29 02:36:43.370 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 02:36:43.385 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 02:36:43.448 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 02:36:43.458 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 03:03:58.373 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 03:03:58.390 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 03:03:58.450 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 03:03:58.466 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 03:33:58.373 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 03:33:58.405 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 03:33:58.466 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 03:33:58.468 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 04:03:58.390 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 04:03:58.422 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 04:03:58.452 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 04:03:58.505 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 04:33:58.387 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 04:33:58.421 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 04:33:59.685 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 04:33:59.686 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:04:13.328 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:04:13.330 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:04:13.331 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 05:04:13.331 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:04:13.331 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task command) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 05:04:13.332 257256 ERROR oslo_service.periodic_task >2018-08-29 05:04:43.345 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:04:43.427 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:04:43.462 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:04:43.513 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:14:36.774 257256 INFO cinder.volume.flows.manager.create_volume [req-76085648-5b36-4fe2-bf93-c88216f9b6bf 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Volume 7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1'} >2018-08-29 05:14:37.379 257256 INFO cinder.volume.flows.manager.create_volume [req-76085648-5b36-4fe2-bf93-c88216f9b6bf 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Volume volume-7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1 (7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1): created successfully >2018-08-29 05:14:37.383 257256 INFO cinder.volume.manager [req-76085648-5b36-4fe2-bf93-c88216f9b6bf 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Created volume successfully. >2018-08-29 05:14:54.564 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-8dc1e2ea-5e5e-43c4-b8d3-cb032d70124b 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] initialize_connection: Did not find a preferred node for volume volume-7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1. >2018-08-29 05:14:54.834 257256 INFO cinder.volume.manager [req-8dc1e2ea-5e5e-43c4-b8d3-cb032d70124b 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Initialize volume connection completed successfully. >2018-08-29 05:15:00.398 257256 INFO cinder.volume.manager [req-8f5de229-6ff8-488e-9ca9-18945c94d2f6 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Attaching volume 7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1 to instance 076a1549-569c-4b41-b45d-7f10b22984b6 at mountpoint /dev/vdb on host None. >2018-08-29 05:15:00.468 257256 INFO cinder.volume.manager [req-8f5de229-6ff8-488e-9ca9-18945c94d2f6 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Attach volume completed successfully. >2018-08-29 05:15:02.905 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-b03c0d03-a5ef-4630-b26d-55f167838e28 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Unable to get remote copy information for volume volume-7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1 >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova [req-b03c0d03-a5ef-4630-b26d-55f167838e28 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Failed to notify nova on events: [{'tag': '7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1', 'name': 'volume-extended', 'server_uuid': '076a1549-569c-4b41-b45d-7f10b22984b6'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-1e72423f-b5d0-4faa-b12f-9e2119b74afc) >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova return_raw=True) >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-1e72423f-b5d0-4faa-b12f-9e2119b74afc) >2018-08-29 05:15:03.750 257256 ERROR cinder.compute.nova >2018-08-29 05:15:03.776 257256 INFO cinder.volume.manager [req-b03c0d03-a5ef-4630-b26d-55f167838e28 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Extend volume completed successfully. >2018-08-29 05:20:15.305 257256 INFO cinder.volume.manager [req-66ec2da8-cf70-4fb8-9497-50738223fba1 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Terminate volume connection completed successfully. >2018-08-29 05:20:16.386 257256 INFO cinder.volume.manager [req-011766e9-3a66-4759-bd4e-708576f217b6 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Detaching volume 7fa8cc31-12a2-4cd9-92f0-a0bcf543a0e1 from instance 076a1549-569c-4b41-b45d-7f10b22984b6. >2018-08-29 05:20:16.474 257256 INFO cinder.volume.manager [req-011766e9-3a66-4759-bd4e-708576f217b6 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Detach volume completed successfully. >2018-08-29 05:20:20.502 257256 INFO cinder.volume.manager [req-9d3303e6-5240-419e-9e98-7455e8c3b22c 436e14b4f05e410b8a5e814cd2ad2432 68e1ea52e2fe4b7884b2ae264f7f9eaa - default default] Deleted volume successfully. >2018-08-29 05:21:44.206 257256 INFO cinder.volume.flows.manager.create_volume [req-b4d77fc1-0a2d-4a6f-8330-dcf8d5fe5a68 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Volume 2d8cfab6-8bfd-45f0-8a9e-616cac98842c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2d8cfab6-8bfd-45f0-8a9e-616cac98842c'} >2018-08-29 05:21:44.802 257256 INFO cinder.volume.flows.manager.create_volume [req-b4d77fc1-0a2d-4a6f-8330-dcf8d5fe5a68 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Volume volume-2d8cfab6-8bfd-45f0-8a9e-616cac98842c (2d8cfab6-8bfd-45f0-8a9e-616cac98842c): created successfully >2018-08-29 05:21:44.806 257256 INFO cinder.volume.manager [req-b4d77fc1-0a2d-4a6f-8330-dcf8d5fe5a68 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Created volume successfully. >2018-08-29 05:22:00.138 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-09b2bac6-00b5-4345-b84c-9cdf46225857 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] initialize_connection: Did not find a preferred node for volume volume-2d8cfab6-8bfd-45f0-8a9e-616cac98842c. >2018-08-29 05:22:00.405 257256 INFO cinder.volume.manager [req-09b2bac6-00b5-4345-b84c-9cdf46225857 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Initialize volume connection completed successfully. >2018-08-29 05:22:04.876 257256 INFO cinder.volume.manager [req-8c913cdd-b3c6-4d95-96f0-d6cd21b86a46 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Attaching volume 2d8cfab6-8bfd-45f0-8a9e-616cac98842c to instance f47647ca-ddb2-4032-8bb0-7396a3e0c5dc at mountpoint /dev/vdb on host None. >2018-08-29 05:22:04.947 257256 INFO cinder.volume.manager [req-8c913cdd-b3c6-4d95-96f0-d6cd21b86a46 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Attach volume completed successfully. >2018-08-29 05:22:06.590 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-47c2363a-5744-48e5-b3bb-607fcb6e4d7f 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Unable to get remote copy information for volume volume-2d8cfab6-8bfd-45f0-8a9e-616cac98842c >2018-08-29 05:22:07.479 257256 INFO cinder.compute.nova [req-47c2363a-5744-48e5-b3bb-607fcb6e4d7f 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Nova event response: {u'status': u'completed', u'tag': u'2d8cfab6-8bfd-45f0-8a9e-616cac98842c', u'name': u'volume-extended', u'server_uuid': u'f47647ca-ddb2-4032-8bb0-7396a3e0c5dc', u'code': 200}. >2018-08-29 05:22:07.502 257256 INFO cinder.volume.manager [req-47c2363a-5744-48e5-b3bb-607fcb6e4d7f 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Extend volume completed successfully. >2018-08-29 05:22:35.848 257256 INFO cinder.volume.manager [req-5f34b27f-c68b-4a0b-906b-022cc71cd7c5 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Terminate volume connection completed successfully. >2018-08-29 05:22:36.216 257256 INFO cinder.volume.manager [req-0a17af26-6a12-4f1d-b2f3-df872949c4c3 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Detaching volume 2d8cfab6-8bfd-45f0-8a9e-616cac98842c from instance f47647ca-ddb2-4032-8bb0-7396a3e0c5dc. >2018-08-29 05:22:36.294 257256 INFO cinder.volume.manager [req-0a17af26-6a12-4f1d-b2f3-df872949c4c3 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Detach volume completed successfully. >2018-08-29 05:22:40.872 257256 INFO cinder.volume.manager [req-417476b6-7d28-4f5e-827a-429ce99546dd 9e81b823885e440a95beb1d91367095e c408a3d97627443f9e334d3be4eecc13 - default default] Deleted volume successfully. >2018-08-29 05:26:13.567 257256 INFO cinder.volume.flows.manager.create_volume [req-0e28b0e0-3ef9-418a-ab20-37d2b2d49dc6 e9f89c5ba6a94fe5a40d9eb0f04ec14b 86566b31bd114e7792336bfd06029b89 - default default] Volume 0d70ecef-91ac-4f63-ae61-b27c57e29290: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0d70ecef-91ac-4f63-ae61-b27c57e29290'} >2018-08-29 05:26:14.224 257256 INFO cinder.volume.flows.manager.create_volume [req-0e28b0e0-3ef9-418a-ab20-37d2b2d49dc6 e9f89c5ba6a94fe5a40d9eb0f04ec14b 86566b31bd114e7792336bfd06029b89 - default default] Volume volume-0d70ecef-91ac-4f63-ae61-b27c57e29290 (0d70ecef-91ac-4f63-ae61-b27c57e29290): created successfully >2018-08-29 05:26:14.228 257256 INFO cinder.volume.manager [req-0e28b0e0-3ef9-418a-ab20-37d2b2d49dc6 e9f89c5ba6a94fe5a40d9eb0f04ec14b 86566b31bd114e7792336bfd06029b89 - default default] Created volume successfully. >2018-08-29 05:26:25.446 257256 INFO cinder.volume.manager [req-792a11eb-b72d-4a47-af01-65860baf01fb e9f89c5ba6a94fe5a40d9eb0f04ec14b 86566b31bd114e7792336bfd06029b89 - default default] Create snapshot completed successfully >2018-08-29 05:26:34.532 257031 INFO cinder.volume.flows.manager.create_volume [req-e31d439f-b8ae-4e4b-9f14-9b52e7197ab3 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Volume 4e81b036-7133-4bba-a864-971e907c1a8b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4e81b036-7133-4bba-a864-971e907c1a8b'} >2018-08-29 05:26:35.526 257256 INFO cinder.volume.manager [req-68ebc78f-cd68-430b-9ecf-1bb0792d0103 e9f89c5ba6a94fe5a40d9eb0f04ec14b 86566b31bd114e7792336bfd06029b89 - default default] Delete snapshot completed successfully. >2018-08-29 05:26:36.007 257031 INFO cinder.volume.flows.manager.create_volume [req-e31d439f-b8ae-4e4b-9f14-9b52e7197ab3 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Volume volume-4e81b036-7133-4bba-a864-971e907c1a8b (4e81b036-7133-4bba-a864-971e907c1a8b): created successfully >2018-08-29 05:26:36.012 257031 INFO cinder.volume.manager [req-e31d439f-b8ae-4e4b-9f14-9b52e7197ab3 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Created volume successfully. >2018-08-29 05:26:38.821 257256 INFO cinder.volume.manager [req-1606a485-d87d-44cf-bd86-6b65f83ae314 e9f89c5ba6a94fe5a40d9eb0f04ec14b 86566b31bd114e7792336bfd06029b89 - default default] Deleted volume successfully. >2018-08-29 05:26:48.400 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-6fc12f6a-9b8a-410b-92f5-a530eb2442e4 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] initialize_connection: Did not find a preferred node for volume volume-4e81b036-7133-4bba-a864-971e907c1a8b. >2018-08-29 05:26:48.811 257031 INFO cinder.volume.manager [req-6fc12f6a-9b8a-410b-92f5-a530eb2442e4 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Initialize volume connection completed successfully. >2018-08-29 05:27:00.021 257031 INFO cinder.volume.manager [req-9f5bf5a7-65f0-4054-8e45-3068491082a7 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Attaching volume 4e81b036-7133-4bba-a864-971e907c1a8b to instance 7e53ea02-651c-4802-882d-9a8ba7c34908 at mountpoint /dev/vdb on host None. >2018-08-29 05:27:00.108 257031 INFO cinder.volume.manager [req-9f5bf5a7-65f0-4054-8e45-3068491082a7 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Attach volume completed successfully. >2018-08-29 05:27:16.512 257031 INFO cinder.volume.manager [req-5387699e-5705-47c5-8b70-07bd4a0ce526 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Terminate volume connection completed successfully. >2018-08-29 05:27:16.736 257031 INFO cinder.volume.manager [req-642c6da3-65ed-4f15-9152-5efb9c509612 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Detaching volume 4e81b036-7133-4bba-a864-971e907c1a8b from instance 7e53ea02-651c-4802-882d-9a8ba7c34908. >2018-08-29 05:27:16.897 257031 INFO cinder.volume.manager [req-642c6da3-65ed-4f15-9152-5efb9c509612 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Detach volume completed successfully. >2018-08-29 05:27:17.963 257031 INFO cinder.volume.flows.manager.create_volume [req-8a872bd4-e2cd-4d13-b4b7-7ebb726ce004 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Volume b26efcd5-55d6-4682-b946-ecd44a24c9ae: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b26efcd5-55d6-4682-b946-ecd44a24c9ae'} >2018-08-29 05:27:18.576 257031 INFO cinder.volume.flows.manager.create_volume [req-8a872bd4-e2cd-4d13-b4b7-7ebb726ce004 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Volume volume-b26efcd5-55d6-4682-b946-ecd44a24c9ae (b26efcd5-55d6-4682-b946-ecd44a24c9ae): created successfully >2018-08-29 05:27:18.581 257031 INFO cinder.volume.manager [req-8a872bd4-e2cd-4d13-b4b7-7ebb726ce004 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Created volume successfully. >2018-08-29 05:27:22.078 257031 INFO cinder.volume.manager [req-60d79ee3-b1cf-4dd0-b143-be1b96c309d3 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Deleted volume successfully. >2018-08-29 05:27:23.319 257031 INFO cinder.volume.manager [req-34ebb0b2-d66c-4b29-aa62-56d0c70a979b 4939755ca3da484fae02595c540b5b57 a5083aac02984a15bd145467258c9a41 - default default] Deleted volume successfully. >2018-08-29 05:28:21.438 257031 INFO cinder.volume.flows.manager.create_volume [req-77ba212c-337d-48c3-ae67-6181ec692b88 76c27a79e09c40ef8b81d8ddbc39c8c3 aae5591d275143e18ecf86638ad43d38 - default default] Volume c12c5e5e-a792-4015-a299-afda4362c58d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c12c5e5e-a792-4015-a299-afda4362c58d'} >2018-08-29 05:28:22.757 257031 INFO cinder.volume.flows.manager.create_volume [req-77ba212c-337d-48c3-ae67-6181ec692b88 76c27a79e09c40ef8b81d8ddbc39c8c3 aae5591d275143e18ecf86638ad43d38 - default default] Volume volume-c12c5e5e-a792-4015-a299-afda4362c58d (c12c5e5e-a792-4015-a299-afda4362c58d): created successfully >2018-08-29 05:28:22.761 257031 INFO cinder.volume.manager [req-77ba212c-337d-48c3-ae67-6181ec692b88 76c27a79e09c40ef8b81d8ddbc39c8c3 aae5591d275143e18ecf86638ad43d38 - default default] Created volume successfully. >2018-08-29 05:28:26.966 257031 INFO cinder.volume.manager [req-e30b5fc7-e192-4c34-a015-c19059cd5b36 76c27a79e09c40ef8b81d8ddbc39c8c3 aae5591d275143e18ecf86638ad43d38 - default default] Deleted volume successfully. >2018-08-29 05:28:27.212 257031 INFO cinder.volume.flows.manager.create_volume [req-30b0c84c-f64c-49f4-83a6-a988574267f9 e4a93ec36bac4d36a1cf519c3f585dd9 1a37f3ee36d74bd884ecbf2790ddfbe4 - default default] Volume e6384160-c5cb-4dd5-a445-0ed37364ee5e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e6384160-c5cb-4dd5-a445-0ed37364ee5e'} >2018-08-29 05:28:27.805 257031 INFO cinder.volume.flows.manager.create_volume [req-30b0c84c-f64c-49f4-83a6-a988574267f9 e4a93ec36bac4d36a1cf519c3f585dd9 1a37f3ee36d74bd884ecbf2790ddfbe4 - default default] Volume volume-e6384160-c5cb-4dd5-a445-0ed37364ee5e (e6384160-c5cb-4dd5-a445-0ed37364ee5e): created successfully >2018-08-29 05:28:27.809 257031 INFO cinder.volume.manager [req-30b0c84c-f64c-49f4-83a6-a988574267f9 e4a93ec36bac4d36a1cf519c3f585dd9 1a37f3ee36d74bd884ecbf2790ddfbe4 - default default] Created volume successfully. >2018-08-29 05:28:31.164 257256 INFO cinder.volume.flows.manager.create_volume [req-9f5a772f-79da-42f8-a0a9-6126a15ef1b1 2259bdf31aa44994ba42a06a274e81c6 f2bfb09a11be4f36b7b28d2fe025645f - default default] Volume d6219a4d-6e07-4b45-a5a7-a9a089ede008: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d6219a4d-6e07-4b45-a5a7-a9a089ede008'} >2018-08-29 05:28:31.823 257256 INFO cinder.volume.flows.manager.create_volume [req-9f5a772f-79da-42f8-a0a9-6126a15ef1b1 2259bdf31aa44994ba42a06a274e81c6 f2bfb09a11be4f36b7b28d2fe025645f - default default] Volume volume-d6219a4d-6e07-4b45-a5a7-a9a089ede008 (d6219a4d-6e07-4b45-a5a7-a9a089ede008): created successfully >2018-08-29 05:28:31.827 257256 INFO cinder.volume.manager [req-9f5a772f-79da-42f8-a0a9-6126a15ef1b1 2259bdf31aa44994ba42a06a274e81c6 f2bfb09a11be4f36b7b28d2fe025645f - default default] Created volume successfully. >2018-08-29 05:28:47.078 257256 INFO cinder.volume.flows.manager.create_volume [req-bbe45c39-ebf1-4b0d-87c3-acb3c2cbcd8b de255b7791d544e3884152cbcdeed760 7a7b92a65ef44193a82d372e97bb618f - default default] Volume 7d062a71-04ff-4b7b-8914-2ff0ef970965: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7d062a71-04ff-4b7b-8914-2ff0ef970965'} >2018-08-29 05:28:47.706 257256 INFO cinder.volume.flows.manager.create_volume [req-bbe45c39-ebf1-4b0d-87c3-acb3c2cbcd8b de255b7791d544e3884152cbcdeed760 7a7b92a65ef44193a82d372e97bb618f - default default] Volume volume-7d062a71-04ff-4b7b-8914-2ff0ef970965 (7d062a71-04ff-4b7b-8914-2ff0ef970965): created successfully >2018-08-29 05:28:47.710 257256 INFO cinder.volume.manager [req-bbe45c39-ebf1-4b0d-87c3-acb3c2cbcd8b de255b7791d544e3884152cbcdeed760 7a7b92a65ef44193a82d372e97bb618f - default default] Created volume successfully. >2018-08-29 05:28:51.596 257256 INFO cinder.volume.manager [req-203fe311-81e9-4951-805d-26b7dea08e3d de255b7791d544e3884152cbcdeed760 7a7b92a65ef44193a82d372e97bb618f - default default] Deleted volume successfully. >2018-08-29 05:28:52.218 257031 INFO cinder.volume.flows.manager.create_volume [req-99b3e3df-e8ad-4c80-81cb-940cc080b9e7 bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] Volume bbecb3a2-3beb-4f4f-b7c4-b78a0c5fc4ea: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bbecb3a2-3beb-4f4f-b7c4-b78a0c5fc4ea'} >2018-08-29 05:28:52.874 257031 INFO cinder.volume.flows.manager.create_volume [req-99b3e3df-e8ad-4c80-81cb-940cc080b9e7 bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] Volume volume-bbecb3a2-3beb-4f4f-b7c4-b78a0c5fc4ea (bbecb3a2-3beb-4f4f-b7c4-b78a0c5fc4ea): created successfully >2018-08-29 05:28:52.878 257031 INFO cinder.volume.manager [req-99b3e3df-e8ad-4c80-81cb-940cc080b9e7 bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] Created volume successfully. >2018-08-29 05:29:02.625 257031 INFO cinder.volume.flows.manager.create_volume [req-2b572602-077b-4f2b-92dc-7ad55e2d4fae cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Volume 245567bf-2895-48c6-9f94-81c34e72bd0c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-245567bf-2895-48c6-9f94-81c34e72bd0c'} >2018-08-29 05:29:02.661 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:29:02.749 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:29:03.653 257031 INFO cinder.volume.flows.manager.create_volume [req-2b572602-077b-4f2b-92dc-7ad55e2d4fae cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Volume volume-245567bf-2895-48c6-9f94-81c34e72bd0c (245567bf-2895-48c6-9f94-81c34e72bd0c): created successfully >2018-08-29 05:29:03.657 257031 INFO cinder.volume.manager [req-2b572602-077b-4f2b-92dc-7ad55e2d4fae cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Created volume successfully. >2018-08-29 05:29:05.085 257256 INFO cinder.volume.flows.manager.create_volume [req-afa941bc-c419-4e0b-a350-6c0d560e2f7d cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Volume b2871170-1c71-41d3-bebc-f83a51819133: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b2871170-1c71-41d3-bebc-f83a51819133'} >2018-08-29 05:29:06.272 257256 INFO cinder.volume.flows.manager.create_volume [req-afa941bc-c419-4e0b-a350-6c0d560e2f7d cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Volume volume-b2871170-1c71-41d3-bebc-f83a51819133 (b2871170-1c71-41d3-bebc-f83a51819133): created successfully >2018-08-29 05:29:06.276 257256 INFO cinder.volume.manager [req-afa941bc-c419-4e0b-a350-6c0d560e2f7d cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Created volume successfully. >2018-08-29 05:29:07.392 257031 INFO cinder.volume.flows.manager.create_volume [req-f6b4d517-5d13-44a5-9f61-0c274133797e cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Volume 402710e0-a67d-43dd-8e2a-9512884a3d37: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-402710e0-a67d-43dd-8e2a-9512884a3d37'} >2018-08-29 05:29:07.406 257031 INFO cinder.volume.manager [req-8be19cde-b297-4b0a-9bf4-e71a492f32b9 e4a93ec36bac4d36a1cf519c3f585dd9 1a37f3ee36d74bd884ecbf2790ddfbe4 - default default] Deleted volume successfully. >2018-08-29 05:29:07.414 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:29:07.492 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:29:07.641 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-59552f2d-d56c-441d-b9a7-b6783af2d39a bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] initialize_connection: Did not find a preferred node for volume volume-bbecb3a2-3beb-4f4f-b7c4-b78a0c5fc4ea. >2018-08-29 05:29:08.478 257031 INFO cinder.volume.flows.manager.create_volume [req-f6b4d517-5d13-44a5-9f61-0c274133797e cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Volume volume-402710e0-a67d-43dd-8e2a-9512884a3d37 (402710e0-a67d-43dd-8e2a-9512884a3d37): created successfully >2018-08-29 05:29:08.482 257031 INFO cinder.volume.manager [req-f6b4d517-5d13-44a5-9f61-0c274133797e cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Created volume successfully. >2018-08-29 05:29:08.691 257031 INFO cinder.volume.manager [req-59552f2d-d56c-441d-b9a7-b6783af2d39a bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] Initialize volume connection completed successfully. >2018-08-29 05:29:11.103 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-59552f2d-d56c-441d-b9a7-b6783af2d39a bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:29:11.381 257031 INFO cinder.volume.manager [req-59552f2d-d56c-441d-b9a7-b6783af2d39a bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] Terminate volume connection completed successfully. >2018-08-29 05:29:15.797 257031 INFO cinder.volume.manager [req-fca85eed-7246-464e-b748-fe4e167f955a cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Deleted volume successfully. >2018-08-29 05:29:15.827 257256 INFO cinder.volume.manager [req-e1daf3d3-fec0-4cd9-a21f-7227d8bd0be2 cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Deleted volume successfully. >2018-08-29 05:29:16.167 257031 INFO cinder.volume.manager [req-98f36ca9-86dc-4048-b950-6ff27304d859 bb45688840484c0bb0331433cbb01f86 f8ecd2fc8c964c4089e4d7acd076c00c - default default] Deleted volume successfully. >2018-08-29 05:29:16.348 257031 INFO cinder.volume.manager [req-60e57b80-e92f-47d4-b53e-d5964a8fd702 cccdff847c444a0fa39775b55f37aab6 79a76ce76b904e2e908404cbfb1db0b4 - default default] Deleted volume successfully. >2018-08-29 05:29:25.756 257256 INFO cinder.volume.flows.manager.create_volume [req-fd6d5307-1cd0-44be-b96d-bd79d779105c ea4a6ad6beea4644884bade896905ac6 3bc59a41fbcf4a8182a9f973e542522a - default default] Volume 79eb4878-7f9e-45b2-a3a8-ed185af074bf: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-79eb4878-7f9e-45b2-a3a8-ed185af074bf'} >2018-08-29 05:29:26.470 257256 INFO cinder.volume.flows.manager.create_volume [req-fd6d5307-1cd0-44be-b96d-bd79d779105c ea4a6ad6beea4644884bade896905ac6 3bc59a41fbcf4a8182a9f973e542522a - default default] Volume volume-79eb4878-7f9e-45b2-a3a8-ed185af074bf (79eb4878-7f9e-45b2-a3a8-ed185af074bf): created successfully >2018-08-29 05:29:26.476 257256 INFO cinder.volume.manager [req-fd6d5307-1cd0-44be-b96d-bd79d779105c ea4a6ad6beea4644884bade896905ac6 3bc59a41fbcf4a8182a9f973e542522a - default default] Created volume successfully. >2018-08-29 05:29:31.526 257031 INFO cinder.volume.flows.manager.create_volume [req-610fb957-479d-420d-8fa3-436fe6a06efa c9b7853092604222995720ac144611d5 d176830eb15642148ae15b77b6ba7c0b - default default] Volume 875c74c4-7a2b-4338-a487-7622635b5569: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-875c74c4-7a2b-4338-a487-7622635b5569'} >2018-08-29 05:29:32.082 257031 INFO cinder.volume.flows.manager.create_volume [req-610fb957-479d-420d-8fa3-436fe6a06efa c9b7853092604222995720ac144611d5 d176830eb15642148ae15b77b6ba7c0b - default default] Volume volume-875c74c4-7a2b-4338-a487-7622635b5569 (875c74c4-7a2b-4338-a487-7622635b5569): created successfully >2018-08-29 05:29:32.087 257031 INFO cinder.volume.manager [req-610fb957-479d-420d-8fa3-436fe6a06efa c9b7853092604222995720ac144611d5 d176830eb15642148ae15b77b6ba7c0b - default default] Created volume successfully. >2018-08-29 05:29:32.888 257256 INFO cinder.volume.flows.manager.create_volume [req-3619f746-a1ff-4bda-9cfe-c75942820392 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume 48e9e14a-2ff6-4cc1-a16b-0ef94d9147b2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-48e9e14a-2ff6-4cc1-a16b-0ef94d9147b2'} >2018-08-29 05:29:33.550 257256 INFO cinder.volume.flows.manager.create_volume [req-3619f746-a1ff-4bda-9cfe-c75942820392 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume volume-48e9e14a-2ff6-4cc1-a16b-0ef94d9147b2 (48e9e14a-2ff6-4cc1-a16b-0ef94d9147b2): created successfully >2018-08-29 05:29:33.555 257256 INFO cinder.volume.manager [req-3619f746-a1ff-4bda-9cfe-c75942820392 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Created volume successfully. >2018-08-29 05:29:35.300 257031 INFO cinder.volume.manager [req-6cb5f8e7-40c0-422c-9d31-aa7fd6e77568 c9b7853092604222995720ac144611d5 d176830eb15642148ae15b77b6ba7c0b - default default] Deleted volume successfully. >2018-08-29 05:29:35.365 257256 INFO cinder.volume.manager [req-5bdb4dc2-5f6c-4c30-86cc-774c5bb7fa5f ea4a6ad6beea4644884bade896905ac6 3bc59a41fbcf4a8182a9f973e542522a - default default] Create snapshot completed successfully >2018-08-29 05:29:37.317 257256 INFO cinder.volume.flows.manager.create_volume [req-813b4e4c-a41c-4a8e-bcbf-df28a1d90ae9 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume 44aac061-4f37-4e1e-8ba3-a43bf89cdeca: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-44aac061-4f37-4e1e-8ba3-a43bf89cdeca'} >2018-08-29 05:29:37.370 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:29:37.445 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:29:39.897 257256 INFO cinder.volume.flows.manager.create_volume [req-813b4e4c-a41c-4a8e-bcbf-df28a1d90ae9 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume volume-44aac061-4f37-4e1e-8ba3-a43bf89cdeca (44aac061-4f37-4e1e-8ba3-a43bf89cdeca): created successfully >2018-08-29 05:29:39.903 257256 INFO cinder.volume.manager [req-813b4e4c-a41c-4a8e-bcbf-df28a1d90ae9 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Created volume successfully. >2018-08-29 05:29:40.296 257256 INFO cinder.volume.flows.manager.create_volume [req-8227204e-318f-4672-aa65-9fdd33d5707a 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Volume 64ca30c4-5e8f-4340-b201-7aa1d84c656e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-64ca30c4-5e8f-4340-b201-7aa1d84c656e'} >2018-08-29 05:29:41.152 257256 INFO cinder.volume.flows.manager.create_volume [req-8227204e-318f-4672-aa65-9fdd33d5707a 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Volume volume-64ca30c4-5e8f-4340-b201-7aa1d84c656e (64ca30c4-5e8f-4340-b201-7aa1d84c656e): created successfully >2018-08-29 05:29:41.156 257256 INFO cinder.volume.manager [req-8227204e-318f-4672-aa65-9fdd33d5707a 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Created volume successfully. >2018-08-29 05:29:41.243 257256 INFO cinder.volume.manager [req-1b6a1bc8-52d3-4968-ace8-8636b313013e ea4a6ad6beea4644884bade896905ac6 3bc59a41fbcf4a8182a9f973e542522a - default default] Delete snapshot completed successfully. >2018-08-29 05:29:44.010 257256 INFO cinder.volume.manager [req-0bc69870-99aa-48f9-a203-962e0f7eff29 ea4a6ad6beea4644884bade896905ac6 3bc59a41fbcf4a8182a9f973e542522a - default default] Deleted volume successfully. >2018-08-29 05:29:44.750 257031 INFO cinder.volume.flows.manager.create_volume [req-2c30dde6-0e93-4095-8ddd-7aee410778be 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume 4d04994c-770d-4ee7-9764-49a9eef30cc8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4d04994c-770d-4ee7-9764-49a9eef30cc8'} >2018-08-29 05:29:45.725 257031 INFO cinder.volume.flows.manager.create_volume [req-2c30dde6-0e93-4095-8ddd-7aee410778be 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume volume-4d04994c-770d-4ee7-9764-49a9eef30cc8 (4d04994c-770d-4ee7-9764-49a9eef30cc8): created successfully >2018-08-29 05:29:45.731 257031 INFO cinder.volume.manager [req-2c30dde6-0e93-4095-8ddd-7aee410778be 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Created volume successfully. >2018-08-29 05:29:48.819 257031 INFO cinder.volume.flows.manager.create_volume [req-aab0a796-2da8-4c9e-86bb-47bd6d991d4c 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume d39e6236-edb3-4234-9ed3-76149e0f2bc7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d39e6236-edb3-4234-9ed3-76149e0f2bc7'} >2018-08-29 05:29:49.587 257256 INFO cinder.volume.flows.manager.create_volume [req-1afd6393-0b93-4650-9cad-1a6a9115b935 26ed2274ceb44551bbf7e73a5e6b999b 07f8e7d839cd490e8c14dbfdb45c253a - default default] Volume 3ac50d69-4e70-4397-8bbf-a8aa6c0ba0d3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3ac50d69-4e70-4397-8bbf-a8aa6c0ba0d3'} >2018-08-29 05:29:49.748 257031 INFO cinder.volume.flows.manager.create_volume [req-aab0a796-2da8-4c9e-86bb-47bd6d991d4c 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Volume volume-d39e6236-edb3-4234-9ed3-76149e0f2bc7 (d39e6236-edb3-4234-9ed3-76149e0f2bc7): created successfully >2018-08-29 05:29:49.753 257031 INFO cinder.volume.manager [req-aab0a796-2da8-4c9e-86bb-47bd6d991d4c 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Created volume successfully. >2018-08-29 05:29:50.233 257256 INFO cinder.volume.flows.manager.create_volume [req-1afd6393-0b93-4650-9cad-1a6a9115b935 26ed2274ceb44551bbf7e73a5e6b999b 07f8e7d839cd490e8c14dbfdb45c253a - default default] Volume volume-3ac50d69-4e70-4397-8bbf-a8aa6c0ba0d3 (3ac50d69-4e70-4397-8bbf-a8aa6c0ba0d3): created successfully >2018-08-29 05:29:50.238 257256 INFO cinder.volume.manager [req-1afd6393-0b93-4650-9cad-1a6a9115b935 26ed2274ceb44551bbf7e73a5e6b999b 07f8e7d839cd490e8c14dbfdb45c253a - default default] Created volume successfully. >2018-08-29 05:29:51.285 257256 INFO cinder.volume.manager [req-e433f648-75f6-4fb8-8b4c-c096f576ca68 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Create snapshot completed successfully >2018-08-29 05:29:51.531 257256 INFO cinder.volume.manager [req-92605027-08e5-4808-bd78-54d64c1382e7 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-403031895: creating >2018-08-29 05:29:51.677 257256 INFO cinder.volume.manager [req-92605027-08e5-4808-bd78-54d64c1382e7 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-403031895: created successfully >2018-08-29 05:29:51.717 257256 INFO cinder.volume.manager [req-92605027-08e5-4808-bd78-54d64c1382e7 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group completed successfully. >2018-08-29 05:29:52.928 257256 INFO cinder.volume.manager [req-ac37a447-d330-468c-b193-6e3dd61f9deb 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-413453358: creating >2018-08-29 05:29:52.960 257256 INFO cinder.volume.manager [req-ac37a447-d330-468c-b193-6e3dd61f9deb 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-413453358: created successfully >2018-08-29 05:29:52.991 257256 INFO cinder.volume.manager [req-ac37a447-d330-468c-b193-6e3dd61f9deb 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Create group completed successfully. >2018-08-29 05:29:53.422 257256 INFO cinder.volume.flows.manager.create_volume [req-1591d509-f2ea-438f-9e2b-23992d387ada 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume 1b7f2423-25f3-469d-971e-122a4a1a1725: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1b7f2423-25f3-469d-971e-122a4a1a1725'} >2018-08-29 05:29:53.797 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:29:53.946 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:29:54.039 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:29:54.079 257256 INFO cinder.volume.flows.manager.create_volume [req-b3756b95-04cd-454e-b2a4-aa92698fb3d9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume b74cf1eb-ecdd-4c69-901b-aa922253f419: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b74cf1eb-ecdd-4c69-901b-aa922253f419'} >2018-08-29 05:29:54.144 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:29:54.191 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:29:54.293 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:29:54.405 257256 WARNING cinder.quota [req-a9964fc8-7b7b-4038-b65f-176c4fe034cd 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeMultiBackendTest-Type-959444853 is set by the default quota flag: quota_volumes_tempest-VolumeMultiBackendTest-Type-959444853, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:54.406 257256 WARNING cinder.quota [req-a9964fc8-7b7b-4038-b65f-176c4fe034cd 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeMultiBackendTest-Type-959444853 is set by the default quota flag: quota_gigabytes_tempest-VolumeMultiBackendTest-Type-959444853, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:55.060 257256 INFO cinder.volume.flows.manager.create_volume [req-1591d509-f2ea-438f-9e2b-23992d387ada 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume volume-1b7f2423-25f3-469d-971e-122a4a1a1725 (1b7f2423-25f3-469d-971e-122a4a1a1725): created successfully >2018-08-29 05:29:55.064 257256 INFO cinder.volume.manager [req-1591d509-f2ea-438f-9e2b-23992d387ada 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Created volume successfully. >2018-08-29 05:29:56.512 257256 INFO cinder.volume.manager [req-a9964fc8-7b7b-4038-b65f-176c4fe034cd 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deleted volume successfully. >2018-08-29 05:29:57.378 257256 INFO cinder.volume.flows.manager.create_volume [req-b3756b95-04cd-454e-b2a4-aa92698fb3d9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume volume-b74cf1eb-ecdd-4c69-901b-aa922253f419 (b74cf1eb-ecdd-4c69-901b-aa922253f419): created successfully >2018-08-29 05:29:57.384 257256 INFO cinder.volume.manager [req-b3756b95-04cd-454e-b2a4-aa92698fb3d9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Created volume successfully. >2018-08-29 05:29:57.596 257256 INFO cinder.volume.flows.manager.create_volume [req-76f2f7bf-38d1-4aa3-8f15-70f0ad832961 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Volume 95596ffa-f52a-401f-86d7-151794ba0d17: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-95596ffa-f52a-401f-86d7-151794ba0d17'} >2018-08-29 05:29:58.154 257256 WARNING cinder.quota [req-6905ab9e-f45a-4180-b872-d72d2497fd92 26ed2274ceb44551bbf7e73a5e6b999b 07f8e7d839cd490e8c14dbfdb45c253a - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesNegativeTest-volume-type-1438733056 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesNegativeTest-volume-type-1438733056, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:58.155 257256 WARNING cinder.quota [req-6905ab9e-f45a-4180-b872-d72d2497fd92 26ed2274ceb44551bbf7e73a5e6b999b 07f8e7d839cd490e8c14dbfdb45c253a - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesNegativeTest-volume-type-1438733056 is set by the default quota flag: quota_volumes_tempest-VolumeTypesNegativeTest-volume-type-1438733056, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:58.615 257031 WARNING cinder.quota [req-1b44f074-d6cb-425a-945a-151fa7f75132 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeMultiBackendTest-Type-740645464 is set by the default quota flag: quota_gigabytes_tempest-VolumeMultiBackendTest-Type-740645464, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:58.616 257031 WARNING cinder.quota [req-1b44f074-d6cb-425a-945a-151fa7f75132 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeMultiBackendTest-Type-740645464 is set by the default quota flag: quota_volumes_tempest-VolumeMultiBackendTest-Type-740645464, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:58.975 257256 INFO cinder.volume.manager [req-746f8484-e589-422e-9675-4e9d2153b599 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] GroupSnapshot ab2ceef9-ee66-4b5d-a9ce-8415901774cd: creating. >2018-08-29 05:29:59.023 257031 WARNING cinder.quota [req-ff0cde04-645e-4f3a-828c-9bcbe466bb6b 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeMultiBackendTest-Type-356590154 is set by the default quota flag: quota_volumes_tempest-VolumeMultiBackendTest-Type-356590154, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:59.024 257031 WARNING cinder.quota [req-ff0cde04-645e-4f3a-828c-9bcbe466bb6b 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeMultiBackendTest-Type-356590154 is set by the default quota flag: quota_gigabytes_tempest-VolumeMultiBackendTest-Type-356590154, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:59.249 257256 WARNING cinder.quota [req-0291234b-e691-4c2d-815c-701f47b1521d 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeMultiBackendTest-Type-1410590831 is set by the default quota flag: quota_gigabytes_tempest-VolumeMultiBackendTest-Type-1410590831, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:59.250 257256 WARNING cinder.quota [req-0291234b-e691-4c2d-815c-701f47b1521d 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeMultiBackendTest-Type-1410590831 is set by the default quota flag: quota_volumes_tempest-VolumeMultiBackendTest-Type-1410590831, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:29:59.390 257256 INFO cinder.volume.manager [req-6905ab9e-f45a-4180-b872-d72d2497fd92 26ed2274ceb44551bbf7e73a5e6b999b 07f8e7d839cd490e8c14dbfdb45c253a - default default] Deleted volume successfully. >2018-08-29 05:29:59.731 257031 INFO cinder.volume.manager [req-1b44f074-d6cb-425a-945a-151fa7f75132 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deleted volume successfully. >2018-08-29 05:30:00.290 257031 INFO cinder.volume.manager [req-ff0cde04-645e-4f3a-828c-9bcbe466bb6b 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deleted volume successfully. >2018-08-29 05:30:02.745 257256 INFO cinder.volume.flows.manager.create_volume [req-38756556-9f97-4071-a988-d193b7d39ef2 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Volume 2709475c-aac8-49b7-a749-10debb7a3b34: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2709475c-aac8-49b7-a749-10debb7a3b34'} >2018-08-29 05:30:03.616 257256 INFO cinder.volume.manager [req-0291234b-e691-4c2d-815c-701f47b1521d 9c772bc38e654e8ca7b540e895204c13 1c129e44cca949a9a20001cb5957a9fc - default default] Deleted volume successfully. >2018-08-29 05:30:03.630 257256 INFO cinder.volume.flows.manager.create_volume [req-76f2f7bf-38d1-4aa3-8f15-70f0ad832961 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Volume volume-95596ffa-f52a-401f-86d7-151794ba0d17 (95596ffa-f52a-401f-86d7-151794ba0d17): created successfully >2018-08-29 05:30:03.634 257256 INFO cinder.volume.manager [req-76f2f7bf-38d1-4aa3-8f15-70f0ad832961 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Created volume successfully. >2018-08-29 05:30:04.686 257256 INFO cinder.volume.manager [req-ae2216ad-717a-4fdb-bd47-787db73acd24 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Create snapshot completed successfully >2018-08-29 05:30:04.864 257256 INFO cinder.volume.flows.manager.create_volume [req-38756556-9f97-4071-a988-d193b7d39ef2 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Volume volume-2709475c-aac8-49b7-a749-10debb7a3b34 (2709475c-aac8-49b7-a749-10debb7a3b34): created successfully >2018-08-29 05:30:04.868 257256 INFO cinder.volume.manager [req-38756556-9f97-4071-a988-d193b7d39ef2 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Created volume successfully. >2018-08-29 05:30:09.233 257256 INFO cinder.volume.manager [req-746f8484-e589-422e-9675-4e9d2153b599 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] group_snapshot ab2ceef9-ee66-4b5d-a9ce-8415901774cd: created successfully >2018-08-29 05:30:09.779 257256 INFO cinder.volume.manager [req-e9fb6fc7-dbaf-4e2d-a351-bbdf217b1957 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Create snapshot completed successfully >2018-08-29 05:30:09.948 257256 INFO cinder.volume.manager [req-de104d68-9c3b-44d4-bbfc-992aa731673b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] group_snapshot ab2ceef9-ee66-4b5d-a9ce-8415901774cd: deleting >2018-08-29 05:30:10.300 257256 INFO cinder.volume.manager [req-469fa8e7-f146-47bd-9062-692a71960ff3 e213679c08da4a36907c5cfad6ba3593 77e06418bbf442ba9f6eedc3176d1cb7 - default default] Delete snapshot completed successfully. >2018-08-29 05:30:11.405 257256 INFO cinder.volume.manager [req-2e8c1af1-f638-4225-b70e-82cb546364ac 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group from source-group-0cd8cac1-2683-45aa-baea-8cbb513bf539 completed successfully. >2018-08-29 05:30:15.602 257256 WARNING cinder.quota [req-de104d68-9c3b-44d4-bbfc-992aa731673b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-1061271661 is set by the default quota flag: quota_snapshots_tempest-volume-type-1061271661, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:30:15.789 257256 INFO cinder.volume.manager [req-de104d68-9c3b-44d4-bbfc-992aa731673b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] group_snapshot ab2ceef9-ee66-4b5d-a9ce-8415901774cd: deleted successfully >2018-08-29 05:30:15.860 257256 INFO cinder.volume.manager [req-9e6aadae-3c04-4604-9f28-e8506586573c 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Create snapshot completed successfully >2018-08-29 05:30:16.374 257256 INFO cinder.volume.manager [req-0de205a4-9a82-4c60-beee-60b832138148 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Create snapshot completed successfully >2018-08-29 05:30:16.957 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-5522d5c3-aa79-42ed-8ad3-0e7e28b69f31 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] initialize_connection: Did not find a preferred node for volume volume-2709475c-aac8-49b7-a749-10debb7a3b34. >2018-08-29 05:30:17.406 257256 INFO cinder.volume.manager [req-5522d5c3-aa79-42ed-8ad3-0e7e28b69f31 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Initialize volume connection completed successfully. >2018-08-29 05:30:18.229 257256 WARNING cinder.quota [req-4320d299-eba6-49d8-aa5d-475ae6c41c20 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-1613903185 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-1613903185, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:30:18.230 257256 WARNING cinder.quota [req-4320d299-eba6-49d8-aa5d-475ae6c41c20 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-1613903185 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-1613903185, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:30:19.490 257256 INFO cinder.volume.manager [req-4320d299-eba6-49d8-aa5d-475ae6c41c20 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group completed successfully. >2018-08-29 05:30:19.530 257031 INFO cinder.volume.flows.manager.create_volume [req-b45f29bf-2958-4317-bb04-fa9c0d2eb9bd a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume 8e6b4b83-9e33-42d0-9161-572d623e0688: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8e6b4b83-9e33-42d0-9161-572d623e0688'} >2018-08-29 05:30:24.192 257256 WARNING cinder.quota [req-cafa0e35-84e3-479d-b619-253b87df115f 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1061271661 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1061271661, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:30:24.195 257256 WARNING cinder.quota [req-cafa0e35-84e3-479d-b619-253b87df115f 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1061271661 is set by the default quota flag: quota_volumes_tempest-volume-type-1061271661, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:30:24.297 257031 INFO cinder.volume.flows.manager.create_volume [req-b45f29bf-2958-4317-bb04-fa9c0d2eb9bd a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume volume-8e6b4b83-9e33-42d0-9161-572d623e0688 (8e6b4b83-9e33-42d0-9161-572d623e0688): created successfully >2018-08-29 05:30:24.302 257031 INFO cinder.volume.manager [req-b45f29bf-2958-4317-bb04-fa9c0d2eb9bd a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Created volume successfully. >2018-08-29 05:30:25.093 257031 INFO cinder.volume.manager [req-ed15ee2c-9a68-4815-8768-1684e748b172 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Attaching volume 8e6b4b83-9e33-42d0-9161-572d623e0688 to instance 62182b0f-f703-424e-9c24-659f816f54a7 at mountpoint /dev/vdb on host None. >2018-08-29 05:30:25.172 257256 INFO cinder.volume.manager [req-ea73cd03-e808-46eb-ab14-c2a27526db15 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Attaching volume 2709475c-aac8-49b7-a749-10debb7a3b34 to instance c24b60cb-b019-448e-9e5d-8250b00f0575 at mountpoint /dev/vdb on host None. >2018-08-29 05:30:25.196 257031 INFO cinder.volume.manager [req-ed15ee2c-9a68-4815-8768-1684e748b172 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Attach volume completed successfully. >2018-08-29 05:30:25.259 257256 INFO cinder.volume.manager [req-ea73cd03-e808-46eb-ab14-c2a27526db15 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Attach volume completed successfully. >2018-08-29 05:30:25.345 257031 INFO cinder.volume.manager [req-b70ee2f6-09bd-4394-bcd0-6ce869a26f7e 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-817849523: creating >2018-08-29 05:30:25.378 257031 INFO cinder.volume.manager [req-b70ee2f6-09bd-4394-bcd0-6ce869a26f7e 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-817849523: created successfully >2018-08-29 05:30:25.390 257256 INFO cinder.volume.manager [req-cafa0e35-84e3-479d-b619-253b87df115f 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Delete group completed successfully. >2018-08-29 05:30:25.402 257031 INFO cinder.volume.manager [req-b70ee2f6-09bd-4394-bcd0-6ce869a26f7e 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Create group completed successfully. >2018-08-29 05:30:26.148 257031 INFO cinder.volume.flows.manager.create_volume [req-74759e09-c9b7-488c-a557-e56de2ee06ec 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume c1b76e6a-47bc-4a96-8a92-54a8308586a4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c1b76e6a-47bc-4a96-8a92-54a8308586a4'} >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager [req-babc2a3f-8ac2-4ddf-ad20-2262dbfa8d17 aa2329d8de3b4cbd893f7b163d9a1847 d8511752003246c49e0f16207a51567c - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager force=force) >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 254, in terminate_connection >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-29 05:30:26.855 257031 ERROR cinder.volume.manager >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server [req-babc2a3f-8ac2-4ddf-ad20-2262dbfa8d17 aa2329d8de3b4cbd893f7b163d9a1847 d8511752003246c49e0f16207a51567c - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-29 05:30:26.857 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:30:27.015 257031 INFO cinder.volume.manager [req-b56e191b-1fa4-4926-bafe-652424420a06 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Detaching volume 8e6b4b83-9e33-42d0-9161-572d623e0688 from instance 62182b0f-f703-424e-9c24-659f816f54a7. >2018-08-29 05:30:27.193 257031 INFO cinder.volume.manager [req-b56e191b-1fa4-4926-bafe-652424420a06 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Detach volume completed successfully. >2018-08-29 05:30:27.233 257256 INFO cinder.volume.manager [req-e93ea9c8-142b-4130-98a8-3c02dc2829e6 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Delete snapshot completed successfully. >2018-08-29 05:30:27.326 257256 INFO cinder.volume.manager [req-fe0eac19-9386-4b9f-a9da-52ca21b15188 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group completed successfully. >2018-08-29 05:30:27.463 257256 INFO cinder.volume.manager [req-aa5839d1-11dc-4177-bc10-3ca8bb8de448 e213679c08da4a36907c5cfad6ba3593 77e06418bbf442ba9f6eedc3176d1cb7 - default default] Delete snapshot completed successfully. >2018-08-29 05:30:28.001 257256 INFO cinder.volume.manager [req-16d596fe-a620-4007-aaa9-82d1e8da1fac 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-2052926991: creating >2018-08-29 05:30:28.061 257256 INFO cinder.volume.manager [req-16d596fe-a620-4007-aaa9-82d1e8da1fac 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-2052926991: created successfully >2018-08-29 05:30:28.087 257031 INFO cinder.volume.flows.manager.create_volume [req-74759e09-c9b7-488c-a557-e56de2ee06ec 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume volume-c1b76e6a-47bc-4a96-8a92-54a8308586a4 (c1b76e6a-47bc-4a96-8a92-54a8308586a4): created successfully >2018-08-29 05:30:28.092 257031 INFO cinder.volume.manager [req-74759e09-c9b7-488c-a557-e56de2ee06ec 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Created volume successfully. >2018-08-29 05:30:28.099 257256 INFO cinder.volume.manager [req-16d596fe-a620-4007-aaa9-82d1e8da1fac 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group completed successfully. >2018-08-29 05:30:29.648 257256 INFO cinder.volume.flows.manager.create_volume [req-045d8ef8-97cb-4aa1-8495-4b1773397577 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume c6f9cbe4-0f72-43e0-83a6-0a83d1183cd2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c6f9cbe4-0f72-43e0-83a6-0a83d1183cd2'} >2018-08-29 05:30:31.870 257256 INFO cinder.volume.flows.manager.create_volume [req-045d8ef8-97cb-4aa1-8495-4b1773397577 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume volume-c6f9cbe4-0f72-43e0-83a6-0a83d1183cd2 (c6f9cbe4-0f72-43e0-83a6-0a83d1183cd2): created successfully >2018-08-29 05:30:31.875 257256 INFO cinder.volume.manager [req-045d8ef8-97cb-4aa1-8495-4b1773397577 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Created volume successfully. >2018-08-29 05:30:32.212 257031 WARNING cinder.quota [req-cad8eb74-d554-45bb-99cd-35646d24eaac 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1023388478 is set by the default quota flag: quota_volumes_tempest-volume-type-1023388478, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:30:32.213 257031 WARNING cinder.quota [req-cad8eb74-d554-45bb-99cd-35646d24eaac 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1023388478 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1023388478, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:30:32.736 257256 INFO cinder.volume.flows.manager.create_volume [req-547c08d4-c753-4aad-abe0-558555667de3 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume def45075-c001-4ba2-80ae-ab1d5f2b315c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-def45075-c001-4ba2-80ae-ab1d5f2b315c'} >2018-08-29 05:30:32.848 257256 INFO cinder.volume.manager [req-6956833e-8d07-48db-995b-52158752de2b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-2104115794: creating >2018-08-29 05:30:32.875 257256 INFO cinder.volume.manager [req-6956833e-8d07-48db-995b-52158752de2b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-2104115794: created successfully >2018-08-29 05:30:32.901 257256 INFO cinder.volume.manager [req-6956833e-8d07-48db-995b-52158752de2b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Create group completed successfully. >2018-08-29 05:30:33.136 257256 INFO cinder.volume.manager [req-43a6c12d-794a-4134-9b1a-f092a6cbb10b 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] GroupSnapshot eae7e943-76e5-4527-897a-41fbbf7f5da9: creating. >2018-08-29 05:30:33.512 257256 INFO cinder.volume.flows.manager.create_volume [req-9d712073-8143-405d-be0a-ffca4b00cef8 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume 867a53fa-c134-4545-bad8-96a391e81475: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-867a53fa-c134-4545-bad8-96a391e81475'} >2018-08-29 05:30:34.975 257031 INFO cinder.volume.manager [req-cad8eb74-d554-45bb-99cd-35646d24eaac 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Delete group completed successfully. >2018-08-29 05:30:35.106 257256 INFO cinder.volume.manager [req-e93ea9c8-142b-4130-98a8-3c02dc2829e6 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Delete snapshot completed successfully. >2018-08-29 05:30:36.279 257256 INFO cinder.volume.flows.manager.create_volume [req-547c08d4-c753-4aad-abe0-558555667de3 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume volume-def45075-c001-4ba2-80ae-ab1d5f2b315c (def45075-c001-4ba2-80ae-ab1d5f2b315c): created successfully >2018-08-29 05:30:36.285 257256 INFO cinder.volume.manager [req-547c08d4-c753-4aad-abe0-558555667de3 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Created volume successfully. >2018-08-29 05:30:36.410 257256 INFO cinder.volume.manager [req-8dab905c-cd25-45eb-923a-3db468bf7b2e 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Create snapshot completed successfully >2018-08-29 05:30:36.660 257256 INFO cinder.volume.flows.manager.create_volume [req-9d712073-8143-405d-be0a-ffca4b00cef8 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume volume-867a53fa-c134-4545-bad8-96a391e81475 (867a53fa-c134-4545-bad8-96a391e81475): created successfully >2018-08-29 05:30:36.664 257256 INFO cinder.volume.manager [req-9d712073-8143-405d-be0a-ffca4b00cef8 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Created volume successfully. >2018-08-29 05:30:37.076 257256 INFO cinder.volume.manager [req-28693c6c-5af1-488d-bdf0-33fb5168f09b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] GroupSnapshot b9cc27a1-a661-4d6b-acad-33a41f0de70a: creating. >2018-08-29 05:30:40.275 257031 INFO cinder.volume.flows.manager.create_volume [req-ad010be6-cbd8-4b0e-8725-16270f5f7de6 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Volume 4bad3f88-2c59-411c-b645-fbbd157f47c0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4bad3f88-2c59-411c-b645-fbbd157f47c0'} >2018-08-29 05:30:41.375 257256 INFO cinder.volume.manager [req-2f3c6ada-cb8d-413b-85c1-648da4290144 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Terminate volume connection completed successfully. >2018-08-29 05:30:41.479 257256 INFO cinder.volume.manager [req-e93ea9c8-142b-4130-98a8-3c02dc2829e6 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Deleted volume successfully. >2018-08-29 05:30:41.960 257256 INFO cinder.volume.manager [req-4af8ab28-855f-49f7-ab04-7b3a00c17f49 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Detaching volume 2709475c-aac8-49b7-a749-10debb7a3b34 from instance c24b60cb-b019-448e-9e5d-8250b00f0575. >2018-08-29 05:30:42.084 257256 INFO cinder.volume.manager [req-4af8ab28-855f-49f7-ab04-7b3a00c17f49 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Detach volume completed successfully. >2018-08-29 05:30:42.558 257256 INFO cinder.volume.manager [req-43a6c12d-794a-4134-9b1a-f092a6cbb10b 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] group_snapshot eae7e943-76e5-4527-897a-41fbbf7f5da9: created successfully >2018-08-29 05:30:42.996 257031 INFO cinder.volume.flows.manager.create_volume [req-ad010be6-cbd8-4b0e-8725-16270f5f7de6 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Volume volume-4bad3f88-2c59-411c-b645-fbbd157f47c0 (4bad3f88-2c59-411c-b645-fbbd157f47c0): created successfully >2018-08-29 05:30:43.001 257031 INFO cinder.volume.manager [req-ad010be6-cbd8-4b0e-8725-16270f5f7de6 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Created volume successfully. >2018-08-29 05:30:45.692 257256 INFO cinder.volume.manager [req-a051b2e8-865f-48e2-97ba-101c745f8da1 aa2329d8de3b4cbd893f7b163d9a1847 d8511752003246c49e0f16207a51567c - default default] Deleted volume successfully. >2018-08-29 05:30:46.187 257031 INFO cinder.volume.flows.manager.create_volume [req-6b71aab2-b837-4efc-93d0-5f383b7816db a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume 02737cbe-85af-40d6-a308-ab99c67fba1e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-02737cbe-85af-40d6-a308-ab99c67fba1e'} >2018-08-29 05:30:48.913 257256 INFO cinder.volume.manager [req-cf620b39-3ea4-42fe-8aea-a3a7dab871b5 cd1e9940f51b4aa1894e69b23e0d0de0 d736ec6c4a9c4d3eb66e2a27423288c3 - default default] Deleted volume successfully. >2018-08-29 05:30:48.987 257256 INFO cinder.volume.manager [req-2550330a-6dc5-4cf8-b179-4d7754cc8415 e213679c08da4a36907c5cfad6ba3593 77e06418bbf442ba9f6eedc3176d1cb7 - default default] Delete snapshot completed successfully. >2018-08-29 05:30:49.019 257256 INFO cinder.volume.manager [req-28693c6c-5af1-488d-bdf0-33fb5168f09b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] group_snapshot b9cc27a1-a661-4d6b-acad-33a41f0de70a: created successfully >2018-08-29 05:30:49.096 257031 INFO cinder.volume.flows.manager.create_volume [req-6b71aab2-b837-4efc-93d0-5f383b7816db a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume volume-02737cbe-85af-40d6-a308-ab99c67fba1e (02737cbe-85af-40d6-a308-ab99c67fba1e): created successfully >2018-08-29 05:30:49.100 257031 INFO cinder.volume.manager [req-6b71aab2-b837-4efc-93d0-5f383b7816db a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Created volume successfully. >2018-08-29 05:30:51.928 257031 INFO cinder.volume.flows.manager.create_volume [req-7a5f84ce-528c-412d-bb9a-6dbcaf5f7513 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Volume 4d8f38e6-de46-44ac-944d-36bc2798f1d0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4d8f38e6-de46-44ac-944d-36bc2798f1d0'} >2018-08-29 05:30:54.347 257031 INFO cinder.volume.flows.manager.create_volume [req-7a5f84ce-528c-412d-bb9a-6dbcaf5f7513 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Volume volume-4d8f38e6-de46-44ac-944d-36bc2798f1d0 (4d8f38e6-de46-44ac-944d-36bc2798f1d0): created successfully >2018-08-29 05:30:54.352 257031 INFO cinder.volume.manager [req-7a5f84ce-528c-412d-bb9a-6dbcaf5f7513 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Created volume successfully. >2018-08-29 05:30:55.664 257031 INFO cinder.volume.manager [req-8e4817dc-f612-41d0-ba2f-1cd0944bb782 aa2329d8de3b4cbd893f7b163d9a1847 d8511752003246c49e0f16207a51567c - default default] Deleted volume successfully. >2018-08-29 05:30:55.911 257256 INFO cinder.volume.manager [req-df5d3982-5feb-478c-b733-9ecf61d691f6 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Create snapshot completed successfully >2018-08-29 05:30:56.285 257256 INFO cinder.volume.flows.manager.create_volume [req-7b80600c-187d-4d02-91ad-75f2f4294578 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume ba1873d4-08f2-4da4-9576-a25a9be5af3e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ba1873d4-08f2-4da4-9576-a25a9be5af3e'} >2018-08-29 05:30:56.783 257256 INFO cinder.volume.flows.manager.create_volume [req-cea2caa4-eea7-4347-8956-bfb7ec082ea0 9954f8f5fa40404a89087f9462088a56 cb87c92fef5549f59cb1893f24029be1 - default default] Volume 9da1165e-5594-4736-bea3-7c5d00e1135a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9da1165e-5594-4736-bea3-7c5d00e1135a'} >2018-08-29 05:31:01.203 257031 INFO cinder.volume.manager [req-ccbc788a-1435-4f88-aa7e-e12f27fab7f0 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Create snapshot completed successfully >2018-08-29 05:31:01.785 257256 INFO cinder.volume.manager [req-08746e05-dcc8-452c-8731-38f9dfb6379c 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group from source-snapshot-eae7e943-76e5-4527-897a-41fbbf7f5da9 completed successfully. >2018-08-29 05:31:02.253 257256 INFO cinder.volume.flows.manager.create_volume [req-7b80600c-187d-4d02-91ad-75f2f4294578 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume volume-ba1873d4-08f2-4da4-9576-a25a9be5af3e (ba1873d4-08f2-4da4-9576-a25a9be5af3e): created successfully >2018-08-29 05:31:02.258 257256 INFO cinder.volume.manager [req-7b80600c-187d-4d02-91ad-75f2f4294578 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Created volume successfully. >2018-08-29 05:31:02.376 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-dc429b16-8418-4187-af50-741441723bcf 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Updating status for CG: ed7e4971-af6e-4b07-a1c8-8250dd8ed4d7. >2018-08-29 05:31:02.392 257031 INFO cinder.volume.flows.manager.create_volume [req-7ff0b045-8c28-4409-bc7f-87d8a05efae7 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Volume e4f3b7dc-adf7-4caa-8664-8bfa1cff4dcd: being created as snap with specification: {'status': u'creating', 'volume_size': 1, 'snapshot_id': 'e15bf9cc-602c-44f2-b79b-b235faa66f0c', 'volume_name': 'volume-e4f3b7dc-adf7-4caa-8664-8bfa1cff4dcd'} >2018-08-29 05:31:02.584 257256 INFO cinder.volume.flows.manager.create_volume [req-cea2caa4-eea7-4347-8956-bfb7ec082ea0 9954f8f5fa40404a89087f9462088a56 cb87c92fef5549f59cb1893f24029be1 - default default] Volume volume-9da1165e-5594-4736-bea3-7c5d00e1135a (9da1165e-5594-4736-bea3-7c5d00e1135a): created successfully >2018-08-29 05:31:02.588 257256 INFO cinder.volume.manager [req-cea2caa4-eea7-4347-8956-bfb7ec082ea0 9954f8f5fa40404a89087f9462088a56 cb87c92fef5549f59cb1893f24029be1 - default default] Created volume successfully. >2018-08-29 05:31:02.593 257256 INFO cinder.volume.manager [req-dc429b16-8418-4187-af50-741441723bcf 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Create group from source-snapshot-b9cc27a1-a661-4d6b-acad-33a41f0de70a completed successfully. >2018-08-29 05:31:09.881 257256 INFO cinder.volume.flows.manager.create_volume [req-a2cb5c8d-e2b3-4d82-b77c-5df4c6c45e24 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume fb7bbf01-4b4c-43a7-877e-46769b9899ef: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fb7bbf01-4b4c-43a7-877e-46769b9899ef'} >2018-08-29 05:31:09.983 257256 INFO cinder.volume.manager [req-e9295696-e55c-412b-9f8a-0523ee152dad e213679c08da4a36907c5cfad6ba3593 77e06418bbf442ba9f6eedc3176d1cb7 - default default] Delete snapshot completed successfully. >2018-08-29 05:31:10.682 257256 INFO cinder.volume.manager [req-5063cc4f-1a53-4946-a557-fe32e76cc2e8 aa2329d8de3b4cbd893f7b163d9a1847 d8511752003246c49e0f16207a51567c - default default] Deleted volume successfully. >2018-08-29 05:31:10.765 257256 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 3.08 sec >2018-08-29 05:31:10.866 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Error has occurred: Unexpected error while running command. >Command: svctask rmfcmap -force 4 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 4 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first.\n' >2018-08-29 05:31:10.897 257031 INFO cinder.volume.flows.manager.create_volume [req-7ff0b045-8c28-4409-bc7f-87d8a05efae7 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Volume volume-e4f3b7dc-adf7-4caa-8664-8bfa1cff4dcd (e4f3b7dc-adf7-4caa-8664-8bfa1cff4dcd): created successfully >2018-08-29 05:31:10.901 257031 INFO cinder.volume.manager [req-7ff0b045-8c28-4409-bc7f-87d8a05efae7 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Created volume successfully. >2018-08-29 05:31:11.230 257031 INFO cinder.volume.flows.manager.create_volume [req-0f2e6d41-a1ea-430b-89a3-b778593179e8 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Volume e1030842-8407-4056-aceb-b4a5e0c1a814: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e1030842-8407-4056-aceb-b4a5e0c1a814'} >2018-08-29 05:31:12.334 257256 INFO cinder.volume.flows.manager.create_volume [req-a2cb5c8d-e2b3-4d82-b77c-5df4c6c45e24 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume volume-fb7bbf01-4b4c-43a7-877e-46769b9899ef (fb7bbf01-4b4c-43a7-877e-46769b9899ef): created successfully >2018-08-29 05:31:12.339 257256 INFO cinder.volume.manager [req-a2cb5c8d-e2b3-4d82-b77c-5df4c6c45e24 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Created volume successfully. >2018-08-29 05:31:12.514 257256 INFO cinder.volume.manager [req-2a8626fb-bbc3-40b7-b0a2-f8b26be56409 9954f8f5fa40404a89087f9462088a56 cb87c92fef5549f59cb1893f24029be1 - default default] Create snapshot completed successfully >2018-08-29 05:31:13.106 257256 WARNING cinder.quota [req-4d984932-eba4-46bc-bafe-86340f2b7d4c 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-302225942 is set by the default quota flag: quota_volumes_tempest-volume-type-302225942, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:31:13.107 257256 WARNING cinder.quota [req-4d984932-eba4-46bc-bafe-86340f2b7d4c 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-302225942 is set by the default quota flag: quota_gigabytes_tempest-volume-type-302225942, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:31:13.840 257031 INFO cinder.volume.flows.manager.create_volume [req-0f2e6d41-a1ea-430b-89a3-b778593179e8 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Volume volume-e1030842-8407-4056-aceb-b4a5e0c1a814 (e1030842-8407-4056-aceb-b4a5e0c1a814): created successfully >2018-08-29 05:31:13.845 257031 INFO cinder.volume.manager [req-0f2e6d41-a1ea-430b-89a3-b778593179e8 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Created volume successfully. >2018-08-29 05:31:13.940 257256 INFO cinder.volume.manager [req-3768c8db-238d-4449-ad3e-2b503300c85f 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] group_snapshot b9cc27a1-a661-4d6b-acad-33a41f0de70a: deleting >2018-08-29 05:31:14.168 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Error running SSH command: svctask rmfcmap -force 4: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 4 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first.\n' >2018-08-29 05:31:14.169 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 4 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first.\n' >2018-08-29 05:31:14.169 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Error running SSH command: svctask rmfcmap -force 4: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 4 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first.\n' >2018-08-29 05:31:14.170 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] CLI Exception output: > command: ['svctask', 'rmfcmap', '-force', u'4'] > stdout: > stderr: CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 4 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first.\n' >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' failed: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'rmfcmap', '-force', u'4'] > stdout: > stderr: CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first. >. >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall Traceback (most recent call last): >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 137, in _run_loop >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw) >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1717, in _check_vdisk_fc_mappings >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall self.ssh.rmfcmap(map_id) >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 564, in rmfcmap >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall self.run_ssh_assert_no_output(ssh_cmd) >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 168, in run_ssh_assert_no_output >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall out, err = self._run_ssh(ssh_cmd) >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall command: ['svctask', 'rmfcmap', '-force', u'4'] >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall stdout: >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall stderr: CMMVC5897E The FlashCopy mapping was not deleted because the mapping or consistency group is in the prepared state. The mapping or consistency group must be stopped first. >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall . >2018-08-29 05:31:14.171 257256 ERROR oslo.service.loopingcall >2018-08-29 05:31:14.229 257256 ERROR cinder.volume.manager [req-7b4fcae6-1655-4923-b88b-2a91b6db88c0 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group failed.: NotImplementedError >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server [req-7b4fcae6-1655-4923-b88b-2a91b6db88c0 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Exception during message handling: VolumeDriverException: Volume driver reported an error: Delete group failed. >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 3338, in delete_group >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server vol_obj.save() >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 3323, in delete_group >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server raise exception.VolumeDriverException(message=msg) >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server VolumeDriverException: Volume driver reported an error: Delete group failed. >2018-08-29 05:31:14.244 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:31:14.344 257256 INFO cinder.volume.manager [req-4d984932-eba4-46bc-bafe-86340f2b7d4c 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Delete group completed successfully. >2018-08-29 05:31:21.666 257031 INFO cinder.volume.flows.manager.create_volume [req-accaf431-e427-42d6-accd-5a7e5a7b26c1 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume 902ebf0c-3a43-490a-8ec5-668f4e7bc4ae: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-902ebf0c-3a43-490a-8ec5-668f4e7bc4ae'} >2018-08-29 05:31:22.175 257256 INFO cinder.volume.manager [req-c56745ac-f1a8-47ea-a223-eaf34400de06 aa2329d8de3b4cbd893f7b163d9a1847 d8511752003246c49e0f16207a51567c - default default] Deleted volume successfully. >2018-08-29 05:31:22.244 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Error has occurred: Unexpected error while running command. >Command: svctask rmfcmap -force 1 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 1 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps.\n' >2018-08-29 05:31:24.298 257256 INFO cinder.volume.manager [req-b5ac7f9e-749b-486c-8512-14672b86cbfa 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Delete snapshot completed successfully. >2018-08-29 05:31:24.461 257031 INFO cinder.volume.flows.manager.create_volume [req-accaf431-e427-42d6-accd-5a7e5a7b26c1 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Volume volume-902ebf0c-3a43-490a-8ec5-668f4e7bc4ae (902ebf0c-3a43-490a-8ec5-668f4e7bc4ae): created successfully >2018-08-29 05:31:24.465 257031 INFO cinder.volume.manager [req-accaf431-e427-42d6-accd-5a7e5a7b26c1 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Created volume successfully. >2018-08-29 05:31:25.382 257256 WARNING cinder.quota [req-3768c8db-238d-4449-ad3e-2b503300c85f 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-302225942 is set by the default quota flag: quota_snapshots_tempest-volume-type-302225942, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:31:25.488 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Error running SSH command: svctask rmfcmap -force 1: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 1 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps.\n' >2018-08-29 05:31:25.489 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 1 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps.\n' >2018-08-29 05:31:25.489 257256 INFO cinder.volume.manager [req-3768c8db-238d-4449-ad3e-2b503300c85f 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] group_snapshot b9cc27a1-a661-4d6b-acad-33a41f0de70a: deleted successfully >2018-08-29 05:31:25.489 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] Error running SSH command: svctask rmfcmap -force 1: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 1 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps.\n' >2018-08-29 05:31:25.490 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [-] CLI Exception output: > command: ['svctask', 'rmfcmap', '-force', u'1'] > stdout: > stderr: CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmfcmap -force 1 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps.\n' >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' failed: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'rmfcmap', '-force', u'1'] > stdout: > stderr: CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps. >. >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall Traceback (most recent call last): >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 137, in _run_loop >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw) >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1709, in _check_vdisk_fc_mappings >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall self.ssh.rmfcmap(map_id) >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 564, in rmfcmap >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall self.run_ssh_assert_no_output(ssh_cmd) >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 168, in run_ssh_assert_no_output >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall out, err = self._run_ssh(ssh_cmd) >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall command: ['svctask', 'rmfcmap', '-force', u'1'] >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall stdout: >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall stderr: CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps. >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall . >2018-08-29 05:31:25.491 257031 ERROR oslo.service.loopingcall >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server [req-35f7b984-896f-468b-814a-6f9f47354fa1 24f4220d00ee4c008f96c2122b4a68d7 8c6eeb58daaf4ce38a5b5950c20923b6 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'rmfcmap', '-force', u'1'] > stdout: > stderr: CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps. >. >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 793, in delete_volume >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server new_status) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 770, in delete_volume >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server self.delete_snapshot(context, s) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_snapshot >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1102, in delete_snapshot >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server snapshot.save() >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1092, in delete_snapshot >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server self.driver.delete_snapshot(snapshot) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2593, in delete_snapshot >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server self._helpers.delete_vdisk(snapshot['name'], False) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1856, in delete_vdisk >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server allow_fctgt=True) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1736, in ensure_vdisk_no_fc_mappings >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server ret = timer.start(interval=self.check_fcmapping_interval).wait() >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server return hubs.get_hub().switch() >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server return self.greenlet.switch() >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 137, in _run_loop >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server result = func(*self.args, **self.kw) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1709, in _check_vdisk_fc_mappings >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server self.ssh.rmfcmap(map_id) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 564, in rmfcmap >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server self.run_ssh_assert_no_output(ssh_cmd) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 168, in run_ssh_assert_no_output >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server out, err = self._run_ssh(ssh_cmd) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server command: ['svctask', 'rmfcmap', '-force', u'1'] >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server stdout: >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server stderr: CMMVC6318E The FlashCopy mapping could not be deleted because the source volume is required by other maps. >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server . >2018-08-29 05:31:25.517 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:31:28.762 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b2e9406d-f53d-4de1-a1aa-06763767472e a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] initialize_connection: Did not find a preferred node for volume volume-e1030842-8407-4056-aceb-b4a5e0c1a814. >2018-08-29 05:31:28.887 257256 INFO cinder.volume.manager [req-ac273a40-d56c-4b12-8ba4-4c3eba4c88fe 9954f8f5fa40404a89087f9462088a56 cb87c92fef5549f59cb1893f24029be1 - default default] Delete snapshot completed successfully. >2018-08-29 05:31:30.338 257031 INFO cinder.volume.manager [req-b2e9406d-f53d-4de1-a1aa-06763767472e a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Initialize volume connection completed successfully. >2018-08-29 05:31:31.234 257256 INFO cinder.volume.manager [req-67841226-171d-4cdb-b734-bc887694be1f 89027704e28940e0afc6e9be3df81350 1e83984a519a41aa9de75ae85bb2a237 - default default] Deleted volume successfully. >2018-08-29 05:31:31.802 257256 INFO cinder.volume.manager [req-02bd508e-0e37-470c-a07c-bbc18d20bbd6 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Delete group completed successfully. >2018-08-29 05:31:31.899 257031 INFO cinder.volume.manager [req-e27069b1-d484-4bc4-ab1c-61304f23f417 a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Deleted volume successfully. >2018-08-29 05:31:32.142 257256 INFO cinder.volume.manager [req-adf61ef4-bd92-4ee8-b910-2a0acead0c3d 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-1738124074: creating >2018-08-29 05:31:32.155 257031 INFO cinder.volume.manager [req-736d77b4-ba78-463a-9a79-4f3dba298bdd a9fc632287ca484e839468fb787826b6 7b1ea197805c4167a5227013e8c455aa - default default] Deleted volume successfully. >2018-08-29 05:31:32.197 257256 INFO cinder.volume.manager [req-adf61ef4-bd92-4ee8-b910-2a0acead0c3d 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Group tempest-CG-1738124074: created successfully >2018-08-29 05:31:32.229 257256 INFO cinder.volume.manager [req-adf61ef4-bd92-4ee8-b910-2a0acead0c3d 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Create group completed successfully. >2018-08-29 05:31:32.909 257256 INFO cinder.volume.flows.manager.create_volume [req-41385225-912a-4ccc-873d-23f86f6afdd9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume 33e988ae-e753-4c08-b701-806a87983dd1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-33e988ae-e753-4c08-b701-806a87983dd1'} >2018-08-29 05:31:33.375 257031 INFO cinder.volume.flows.manager.create_volume [req-134e9234-b461-4fce-a6d6-e7d979bb09a5 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Volume e4ea7870-f04d-4c32-8727-5ceba4a05c82: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e4ea7870-f04d-4c32-8727-5ceba4a05c82'} >2018-08-29 05:31:33.721 257256 INFO cinder.volume.manager [req-8e96d836-0e2a-4537-a45b-58834cbbef0e 9954f8f5fa40404a89087f9462088a56 cb87c92fef5549f59cb1893f24029be1 - default default] Deleted volume successfully. >2018-08-29 05:31:34.435 257256 INFO cinder.volume.flows.manager.create_volume [req-41385225-912a-4ccc-873d-23f86f6afdd9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Volume volume-33e988ae-e753-4c08-b701-806a87983dd1 (33e988ae-e753-4c08-b701-806a87983dd1): created successfully >2018-08-29 05:31:34.439 257256 INFO cinder.volume.manager [req-41385225-912a-4ccc-873d-23f86f6afdd9 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Created volume successfully. >2018-08-29 05:31:34.671 257031 INFO cinder.volume.flows.manager.create_volume [req-134e9234-b461-4fce-a6d6-e7d979bb09a5 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Volume volume-e4ea7870-f04d-4c32-8727-5ceba4a05c82 (e4ea7870-f04d-4c32-8727-5ceba4a05c82): created successfully >2018-08-29 05:31:34.675 257031 INFO cinder.volume.manager [req-134e9234-b461-4fce-a6d6-e7d979bb09a5 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Created volume successfully. >2018-08-29 05:31:35.113 257031 INFO cinder.volume.manager [req-af2f2ac7-c876-4b7b-a3fc-bd8a4fb5d564 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Attaching volume e1030842-8407-4056-aceb-b4a5e0c1a814 to instance 3da1e056-cf8e-475e-a239-b1b2dfd94265 at mountpoint /dev/vdb on host None. >2018-08-29 05:31:35.281 257031 INFO cinder.volume.manager [req-af2f2ac7-c876-4b7b-a3fc-bd8a4fb5d564 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Attach volume completed successfully. >2018-08-29 05:31:41.268 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a28e310d-e78e-418a-9da1-b7178492cf05 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Updating status for CG: 81e4a447-64e6-4e72-9af1-fd2712aea30e. >2018-08-29 05:31:41.477 257256 INFO cinder.volume.manager [req-a28e310d-e78e-418a-9da1-b7178492cf05 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Create group from source-group-6c440b5f-b718-46bf-acf6-f53b53cf4ab4 completed successfully. >2018-08-29 05:31:45.114 257256 INFO cinder.volume.flows.manager.create_volume [req-ef7aa0f8-cb79-4fb0-87ee-9064e44c4618 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Volume 348268c4-e914-45f2-990c-c5b38f797107: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-348268c4-e914-45f2-990c-c5b38f797107'} >2018-08-29 05:31:46.044 257256 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 0.08 sec >2018-08-29 05:31:46.253 257256 INFO cinder.volume.flows.manager.create_volume [req-6125acb9-d319-4256-aabd-4bf84b84bb80 ceaf16cde9d5477fb5d6fc33a1fba0a7 2ac89c45109349e4a55ffca7a8e0e3c0 - default default] Volume b42e2973-efb0-4443-9ce4-171bda86a7b4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b42e2973-efb0-4443-9ce4-171bda86a7b4'} >2018-08-29 05:31:47.095 257256 INFO cinder.volume.flows.manager.create_volume [req-ef7aa0f8-cb79-4fb0-87ee-9064e44c4618 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Volume volume-348268c4-e914-45f2-990c-c5b38f797107 (348268c4-e914-45f2-990c-c5b38f797107): created successfully >2018-08-29 05:31:47.102 257256 INFO cinder.volume.manager [req-ef7aa0f8-cb79-4fb0-87ee-9064e44c4618 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Created volume successfully. >2018-08-29 05:31:47.683 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-4fd6de1c-ff7b-4487-8006-1ba6232a5fb9 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] initialize_connection: Did not find a preferred node for volume volume-e4ea7870-f04d-4c32-8727-5ceba4a05c82. >2018-08-29 05:31:48.237 257031 INFO cinder.volume.manager [req-4fd6de1c-ff7b-4487-8006-1ba6232a5fb9 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Initialize volume connection completed successfully. >2018-08-29 05:31:48.315 257256 INFO cinder.volume.flows.manager.create_volume [req-6125acb9-d319-4256-aabd-4bf84b84bb80 ceaf16cde9d5477fb5d6fc33a1fba0a7 2ac89c45109349e4a55ffca7a8e0e3c0 - default default] Volume volume-b42e2973-efb0-4443-9ce4-171bda86a7b4 (b42e2973-efb0-4443-9ce4-171bda86a7b4): created successfully >2018-08-29 05:31:48.320 257256 INFO cinder.volume.manager [req-6125acb9-d319-4256-aabd-4bf84b84bb80 ceaf16cde9d5477fb5d6fc33a1fba0a7 2ac89c45109349e4a55ffca7a8e0e3c0 - default default] Created volume successfully. >2018-08-29 05:31:48.348 257256 WARNING cinder.quota [req-6803fa16-74d4-426f-8b2f-a6dcd3c13d9b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-884694127 is set by the default quota flag: quota_gigabytes_tempest-volume-type-884694127, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:31:48.349 257256 WARNING cinder.quota [req-6803fa16-74d4-426f-8b2f-a6dcd3c13d9b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-884694127 is set by the default quota flag: quota_volumes_tempest-volume-type-884694127, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:31:49.064 257031 INFO cinder.volume.flows.manager.create_volume [req-2de5905c-9a07-4051-bc63-6e7cdce2e5cc 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Volume 8f484adc-f046-471d-9804-9cfda6a2cfe3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8f484adc-f046-471d-9804-9cfda6a2cfe3'} >2018-08-29 05:31:49.118 257256 INFO cinder.volume.manager [req-6803fa16-74d4-426f-8b2f-a6dcd3c13d9b 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Delete group completed successfully. >2018-08-29 05:31:50.070 257031 INFO cinder.volume.flows.manager.create_volume [req-2de5905c-9a07-4051-bc63-6e7cdce2e5cc 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Volume volume-8f484adc-f046-471d-9804-9cfda6a2cfe3 (8f484adc-f046-471d-9804-9cfda6a2cfe3): created successfully >2018-08-29 05:31:50.076 257031 INFO cinder.volume.manager [req-2de5905c-9a07-4051-bc63-6e7cdce2e5cc 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Created volume successfully. >2018-08-29 05:31:52.793 257256 INFO cinder.volume.manager [req-945b0366-4551-4299-89e1-6e29dec4cb0a 3f03473a19ea467db37bc7afe7977262 abbeca7a1f714520a2b6af9f1c188393 - default default] Delete group completed successfully. >2018-08-29 05:31:53.361 257031 INFO cinder.volume.flows.manager.create_volume [req-b17d431f-9d5f-4441-9671-e92a1b76dd1c 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Volume a90b7bb7-2ff8-4944-a710-f6fb5da7bd6b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a90b7bb7-2ff8-4944-a710-f6fb5da7bd6b'} >2018-08-29 05:31:54.820 257031 INFO cinder.volume.flows.manager.create_volume [req-b17d431f-9d5f-4441-9671-e92a1b76dd1c 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Volume volume-a90b7bb7-2ff8-4944-a710-f6fb5da7bd6b (a90b7bb7-2ff8-4944-a710-f6fb5da7bd6b): created successfully >2018-08-29 05:31:54.825 257031 INFO cinder.volume.manager [req-b17d431f-9d5f-4441-9671-e92a1b76dd1c 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Created volume successfully. >2018-08-29 05:31:55.124 257031 INFO cinder.volume.manager [req-9c5c05ad-767f-4af2-bc96-e47cd99a157c 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Attaching volume e4ea7870-f04d-4c32-8727-5ceba4a05c82 to instance 338c0787-183a-41e4-a268-7f33b90d0a1c at mountpoint /dev/vdb on host None. >2018-08-29 05:31:55.252 257031 INFO cinder.volume.manager [req-9c5c05ad-767f-4af2-bc96-e47cd99a157c 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Attach volume completed successfully. >2018-08-29 05:32:00.170 257256 INFO cinder.volume.manager [req-5dc90586-64db-4ff9-b72b-ccdf9e2c8932 ceaf16cde9d5477fb5d6fc33a1fba0a7 2ac89c45109349e4a55ffca7a8e0e3c0 - default default] Create snapshot completed successfully >2018-08-29 05:32:07.466 257256 INFO cinder.volume.manager [req-1fc63558-4154-447a-bea3-c0052accb597 ceaf16cde9d5477fb5d6fc33a1fba0a7 2ac89c45109349e4a55ffca7a8e0e3c0 - default default] Delete snapshot completed successfully. >2018-08-29 05:32:08.514 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-fd94f200-cc9d-482f-99ef-cb6e147c9aa0 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] initialize_connection: Did not find a preferred node for volume volume-8f484adc-f046-471d-9804-9cfda6a2cfe3. >2018-08-29 05:32:11.385 257031 INFO cinder.volume.manager [req-fd94f200-cc9d-482f-99ef-cb6e147c9aa0 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] Initialize volume connection completed successfully. >2018-08-29 05:32:11.548 257256 INFO cinder.volume.manager [req-cc2c3bc4-08b3-4310-b5d1-784e2e65f431 ceaf16cde9d5477fb5d6fc33a1fba0a7 2ac89c45109349e4a55ffca7a8e0e3c0 - default default] Deleted volume successfully. >2018-08-29 05:32:12.028 257031 INFO cinder.volume.manager [req-124c9120-e51c-40b0-aa20-b1abee38bdd7 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Terminate volume connection completed successfully. >2018-08-29 05:32:12.262 257031 INFO cinder.volume.manager [req-ef9542bc-0eca-4bb3-ab64-c523cedc30e2 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Detaching volume e1030842-8407-4056-aceb-b4a5e0c1a814 from instance 3da1e056-cf8e-475e-a239-b1b2dfd94265. >2018-08-29 05:32:12.402 257031 INFO cinder.volume.manager [req-ef9542bc-0eca-4bb3-ab64-c523cedc30e2 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Detach volume completed successfully. >2018-08-29 05:32:16.639 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-fd94f200-cc9d-482f-99ef-cb6e147c9aa0 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:32:16.662 257031 INFO cinder.volume.manager [req-a2e47b27-16c9-486a-ba17-67bbe4714b16 a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Deleted volume successfully. >2018-08-29 05:32:17.270 257031 INFO cinder.volume.manager [req-fd94f200-cc9d-482f-99ef-cb6e147c9aa0 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] Terminate volume connection completed successfully. >2018-08-29 05:32:19.098 257031 INFO cinder.volume.flows.manager.create_volume [req-dfb46399-ceda-470b-ab0a-1d89884deeea 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Volume 5a19ec2b-fb95-4bfc-aaa7-2a521094ecbb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5a19ec2b-fb95-4bfc-aaa7-2a521094ecbb'} >2018-08-29 05:32:19.138 257031 INFO cinder.volume.manager [req-47a50576-0cf8-4d86-a6bb-a208dbc95e4b a4ca89c672164a7684b0944bdf057875 8d80657cd5064deead82a8cbefcd98a7 - default default] Deleted volume successfully. >2018-08-29 05:32:20.043 257031 INFO cinder.volume.flows.manager.create_volume [req-dfb46399-ceda-470b-ab0a-1d89884deeea 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Volume volume-5a19ec2b-fb95-4bfc-aaa7-2a521094ecbb (5a19ec2b-fb95-4bfc-aaa7-2a521094ecbb): created successfully >2018-08-29 05:32:20.047 257031 INFO cinder.volume.manager [req-dfb46399-ceda-470b-ab0a-1d89884deeea 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Created volume successfully. >2018-08-29 05:32:33.004 257031 INFO cinder.volume.flows.manager.create_volume [req-9b01d439-9f18-4024-8e67-e1db35efdace 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Volume 82c26162-8606-4b7b-91a7-6b60e79ee19e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-82c26162-8606-4b7b-91a7-6b60e79ee19e'} >2018-08-29 05:32:33.878 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-7b42efad-0f41-4648-b1fe-cd6c6b9377df 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] initialize_connection: Did not find a preferred node for volume volume-5a19ec2b-fb95-4bfc-aaa7-2a521094ecbb. >2018-08-29 05:32:34.220 257031 INFO cinder.volume.flows.manager.create_volume [req-9b01d439-9f18-4024-8e67-e1db35efdace 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Volume volume-82c26162-8606-4b7b-91a7-6b60e79ee19e (82c26162-8606-4b7b-91a7-6b60e79ee19e): created successfully >2018-08-29 05:32:34.225 257031 INFO cinder.volume.manager [req-9b01d439-9f18-4024-8e67-e1db35efdace 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Created volume successfully. >2018-08-29 05:32:35.065 257031 INFO cinder.volume.manager [req-7b42efad-0f41-4648-b1fe-cd6c6b9377df 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] Initialize volume connection completed successfully. >2018-08-29 05:32:41.837 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-7b42efad-0f41-4648-b1fe-cd6c6b9377df 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:32:42.325 257031 INFO cinder.volume.manager [req-7b42efad-0f41-4648-b1fe-cd6c6b9377df 29064564ae18487e87b9874de3d71d86 813c33dadf834c04b60a2bc626a6bba4 - default default] Terminate volume connection completed successfully. >2018-08-29 05:32:47.326 257031 INFO cinder.volume.manager [req-0d78f3a8-91ea-4868-8531-a80d8b0c8b7e 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Deleted volume successfully. >2018-08-29 05:32:47.834 257031 INFO cinder.volume.manager [req-9c346ad2-cc57-4525-b940-eaac5e670de6 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Terminate volume connection completed successfully. >2018-08-29 05:32:48.361 257031 INFO cinder.volume.manager [req-56b9d832-e28f-4730-bc96-d1eca7e36f00 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Detaching volume e4ea7870-f04d-4c32-8727-5ceba4a05c82 from instance 338c0787-183a-41e4-a268-7f33b90d0a1c. >2018-08-29 05:32:48.572 257031 INFO cinder.volume.manager [req-56b9d832-e28f-4730-bc96-d1eca7e36f00 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Detach volume completed successfully. >2018-08-29 05:32:48.608 257031 INFO cinder.volume.manager [req-308f3e7a-0e0f-4e1c-bd76-492cceb14f1e 46203876b79d4e07afcddabb1aed851b 0b88c53c3bb3491bb92cc8b8ac1e9d4b - default default] Deleted volume successfully. >2018-08-29 05:32:53.696 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-3dc34819-e5fa-450d-a268-2896a773112b 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] initialize_connection: Did not find a preferred node for volume volume-82c26162-8606-4b7b-91a7-6b60e79ee19e. >2018-08-29 05:32:54.021 257031 INFO cinder.volume.manager [req-3dc34819-e5fa-450d-a268-2896a773112b 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Initialize volume connection completed successfully. >2018-08-29 05:32:54.831 257031 INFO cinder.volume.manager [req-e46c97f3-ef0a-4324-a78b-1555bc6e4fa6 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Group tempest-Group-932370873: creating >2018-08-29 05:32:54.884 257031 INFO cinder.volume.manager [req-e46c97f3-ef0a-4324-a78b-1555bc6e4fa6 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Group tempest-Group-932370873: created successfully >2018-08-29 05:32:54.915 257031 INFO cinder.volume.manager [req-e46c97f3-ef0a-4324-a78b-1555bc6e4fa6 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Create group completed successfully. >2018-08-29 05:32:56.455 257031 INFO cinder.volume.flows.manager.create_volume [req-a45b0e4a-0ca6-4f8d-a9dd-20362adecc2c 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Volume 93009602-b109-4369-8c53-270f3b4f6038: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-93009602-b109-4369-8c53-270f3b4f6038'} >2018-08-29 05:32:57.386 257031 INFO cinder.volume.flows.manager.create_volume [req-a45b0e4a-0ca6-4f8d-a9dd-20362adecc2c 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Volume volume-93009602-b109-4369-8c53-270f3b4f6038 (93009602-b109-4369-8c53-270f3b4f6038): created successfully >2018-08-29 05:32:57.392 257031 INFO cinder.volume.manager [req-a45b0e4a-0ca6-4f8d-a9dd-20362adecc2c 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Created volume successfully. >2018-08-29 05:32:58.851 257031 INFO cinder.volume.manager [req-94a2d776-33bb-454f-9698-0477a256aa91 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] GroupSnapshot ac706fba-f199-4db2-904d-9cd2b556500c: creating. >2018-08-29 05:33:00.002 257031 INFO cinder.volume.flows.manager.create_volume [req-a4a46ee5-afc1-4615-96c6-d1ca3a82b1a8 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Volume 9138881e-ce5f-4742-9fd6-1b5f57731404: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9138881e-ce5f-4742-9fd6-1b5f57731404'} >2018-08-29 05:33:01.208 257031 INFO cinder.volume.manager [req-b2b06a11-d9f5-4842-bbbc-895204670743 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Attaching volume 82c26162-8606-4b7b-91a7-6b60e79ee19e to instance 3b0b563f-7e06-47eb-9d6e-ab0d84389a4a at mountpoint /dev/vdb on host None. >2018-08-29 05:33:01.345 257031 INFO cinder.volume.manager [req-b2b06a11-d9f5-4842-bbbc-895204670743 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Attach volume completed successfully. >2018-08-29 05:33:01.360 257031 INFO cinder.volume.flows.manager.create_volume [req-a4a46ee5-afc1-4615-96c6-d1ca3a82b1a8 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Volume volume-9138881e-ce5f-4742-9fd6-1b5f57731404 (9138881e-ce5f-4742-9fd6-1b5f57731404): created successfully >2018-08-29 05:33:01.364 257031 INFO cinder.volume.manager [req-a4a46ee5-afc1-4615-96c6-d1ca3a82b1a8 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Created volume successfully. >2018-08-29 05:33:03.859 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-13d630ac-ce75-4360-9d5c-7257e68876d2 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Unable to get remote copy information for volume volume-82c26162-8606-4b7b-91a7-6b60e79ee19e >2018-08-29 05:33:05.715 257031 INFO cinder.compute.nova [req-13d630ac-ce75-4360-9d5c-7257e68876d2 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Nova event response: {u'status': u'completed', u'tag': u'82c26162-8606-4b7b-91a7-6b60e79ee19e', u'name': u'volume-extended', u'server_uuid': u'3b0b563f-7e06-47eb-9d6e-ab0d84389a4a', u'code': 200}. >2018-08-29 05:33:05.756 257031 INFO cinder.volume.manager [req-13d630ac-ce75-4360-9d5c-7257e68876d2 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Extend volume completed successfully. >2018-08-29 05:33:06.708 257256 INFO cinder.volume.flows.manager.create_volume [req-c8c838fb-5108-4cab-aba8-a82b602226dd d166a9197a3e409b97161f023290cebc bebe9d15c6c4424cb6e40809e7f8f5c6 - default default] Volume d42002a1-539f-4d68-bb43-a15923bb7295: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d42002a1-539f-4d68-bb43-a15923bb7295'} >2018-08-29 05:33:08.690 257256 INFO cinder.volume.flows.manager.create_volume [req-c8c838fb-5108-4cab-aba8-a82b602226dd d166a9197a3e409b97161f023290cebc bebe9d15c6c4424cb6e40809e7f8f5c6 - default default] Volume volume-d42002a1-539f-4d68-bb43-a15923bb7295 (d42002a1-539f-4d68-bb43-a15923bb7295): created successfully >2018-08-29 05:33:08.696 257256 INFO cinder.volume.manager [req-c8c838fb-5108-4cab-aba8-a82b602226dd d166a9197a3e409b97161f023290cebc bebe9d15c6c4424cb6e40809e7f8f5c6 - default default] Created volume successfully. >2018-08-29 05:33:08.860 257031 INFO cinder.volume.manager [req-94a2d776-33bb-454f-9698-0477a256aa91 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] group_snapshot ac706fba-f199-4db2-904d-9cd2b556500c: created successfully >2018-08-29 05:33:12.457 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error has occurred: Unexpected error while running command. >Command: svcinfo lshost -delim ! "WQ_GEAWS0122_620" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n': ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lshost -delim ! "WQ_GEAWS0122_620" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:14.161 257031 INFO cinder.volume.manager [req-df1e9bde-76fa-4562-a85a-f638d7ffd58a 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] group_snapshot ac706fba-f199-4db2-904d-9cd2b556500c: deleting >2018-08-29 05:33:14.597 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-8bb387d6-a82c-4812-85e5-7785082d69e1 d166a9197a3e409b97161f023290cebc bebe9d15c6c4424cb6e40809e7f8f5c6 - default default] CLI Exception output: > command: ['svcinfo', 'lsvdisk', '-bytes', '-delim', '!', '"volume-d42002a1-539f-4d68-bb43-a15923bb7295"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server [req-8bb387d6-a82c-4812-85e5-7785082d69e1 d166a9197a3e409b97161f023290cebc bebe9d15c6c4424cb6e40809e7f8f5c6 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsvdisk', '-bytes', '-delim', '!', '"volume-d42002a1-539f-4d68-bb43-a15923bb7295"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 793, in delete_volume >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server new_status) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 775, in delete_volume >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2556, in delete_volume >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server self._helpers.delete_vdisk(volume['name'], False) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1852, in delete_vdisk >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server if not self.is_vdisk_defined(vdisk): >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1374, in is_vdisk_defined >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server attrs = self.get_vdisk_attributes(vdisk_name) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1369, in get_vdisk_attributes >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server attrs = self.ssh.lsvdisk(vdisk) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 476, in lsvdisk >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server command: ['svcinfo', 'lsvdisk', '-bytes', '-delim', '!', '"volume-d42002a1-539f-4d68-bb43-a15923bb7295"'] >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server stdout: >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server . >2018-08-29 05:33:14.642 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:15.214 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-92e24624-ffa6-4417-a2bd-7e8d55cfed3f 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] CLI Exception output: > command: ['svcinfo', 'lsvdisk', '-bytes', '-delim', '!', '"volume-a90b7bb7-2ff8-4944-a710-f6fb5da7bd6b"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server [req-92e24624-ffa6-4417-a2bd-7e8d55cfed3f 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsvdisk', '-bytes', '-delim', '!', '"volume-a90b7bb7-2ff8-4944-a710-f6fb5da7bd6b"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 793, in delete_volume >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server new_status) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 775, in delete_volume >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2556, in delete_volume >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server self._helpers.delete_vdisk(volume['name'], False) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1852, in delete_vdisk >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server if not self.is_vdisk_defined(vdisk): >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1374, in is_vdisk_defined >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server attrs = self.get_vdisk_attributes(vdisk_name) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1369, in get_vdisk_attributes >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server attrs = self.ssh.lsvdisk(vdisk) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 476, in lsvdisk >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server command: ['svcinfo', 'lsvdisk', '-bytes', '-delim', '!', '"volume-a90b7bb7-2ff8-4944-a710-f6fb5da7bd6b"'] >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server stdout: >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server . >2018-08-29 05:33:15.251 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:15.344 257256 INFO cinder.volume.flows.manager.create_volume [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Volume 765f9ae2-26db-4094-8cbc-2544c42b56a9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9'} >2018-08-29 05:33:15.392 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-df1e9bde-76fa-4562-a85a-f638d7ffd58a 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] CLI Exception output: > command: ['svcinfo', 'lsvdisk', '-bytes', '-delim', '!', '"snapshot-7b680aa2-9b1d-42e2-99d0-49d056a5f0c2"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.: NotImplementedError >2018-08-29 05:33:15.410 257031 ERROR cinder.volume.manager [req-df1e9bde-76fa-4562-a85a-f638d7ffd58a 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Error occurred when deleting group_snapshot ac706fba-f199-4db2-904d-9cd2b556500c.: NotImplementedError >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server [req-df1e9bde-76fa-4562-a85a-f638d7ffd58a 9691de9dcab54fb482a9361c0c6c37ea 8f7df8937a234e248a588aad889a6cc4 - default default] Exception during message handling: VolumeDriverException: Volume driver reported an error: Error occurred when deleting group_snapshot ac706fba-f199-4db2-904d-9cd2b556500c. >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 3858, in delete_group_snapshot >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server snapshot.save() >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 3843, in delete_group_snapshot >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server raise exception.VolumeDriverException(message=msg) >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server VolumeDriverException: Volume driver reported an error: Error occurred when deleting group_snapshot ac706fba-f199-4db2-904d-9cd2b556500c. >2018-08-29 05:33:15.446 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:15.701 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lshost -delim ! "WQ_GEAWS0122_620": ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lshost -delim ! "WQ_GEAWS0122_620" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:15.702 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lshost -delim ! "WQ_GEAWS0122_620" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:15.703 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lshost -delim ! "WQ_GEAWS0122_620": ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lshost -delim ! "WQ_GEAWS0122_620" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:15.704 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] CLI Exception output: > command: ['svcinfo', 'lshost', '-delim', '!', u'"WQ_GEAWS0122_620"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lshost -delim ! "WQ_GEAWS0122_620" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Driver initialize connection failed (error: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lshost', '-delim', '!', u'"WQ_GEAWS0122_620"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.).: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lshost', '-delim', '!', u'"WQ_GEAWS0122_620"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1680, in initialize_connection >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager conn_info = self.driver.initialize_connection(volume, connector) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 80, in decorator >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager conn_info = initialize_connection(self, *args, **kwargs) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 123, in initialize_connection >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager return _do_initialize_connection_locked() >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager return f(*args, **kwargs) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 122, in _do_initialize_connection_locked >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager return self._do_initialize_connection(volume, connector) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 142, in _do_initialize_connection >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager host_name = self._helpers.get_host_from_connector(connector) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 958, in get_host_from_connector >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager pass >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 946, in get_host_from_connector >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager resp = self.ssh.lshost(host=name) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 255, in lshost >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=with_header) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager command: ['svcinfo', 'lshost', '-delim', '!', u'"WQ_GEAWS0122_620"'] >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager stdout: >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager . >2018-08-29 05:33:15.705 257031 ERROR cinder.volume.manager >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Driver initialize connection failed (error: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lshost', '-delim', '!', u'"WQ_GEAWS0122_620"'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.). >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1688, in initialize_connection >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Driver initialize connection failed (error: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server command: ['svcinfo', 'lshost', '-delim', '!', u'"WQ_GEAWS0122_620"'] >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server stdout: >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server .). >2018-08-29 05:33:15.708 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:16.047 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error has occurred: Unexpected error while running command. >Command: svcinfo lsiogrp -delim ! >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n': ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsiogrp -delim ! >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:16.455 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error has occurred: Unexpected error while running command. >Command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n': ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server [req-22f4b4cb-4a0a-4eba-bb00-66bfc4b6986f 1250b27e63084a23b2f8af075ca364ed abb39767e7aa4881b728cb1c898a27a9 - default default] Exception during message handling: IndexError: list index out of range >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 793, in delete_volume >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server new_status) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 775, in delete_volume >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2556, in delete_volume >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server self._helpers.delete_vdisk(volume['name'], False) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1852, in delete_vdisk >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server if not self.is_vdisk_defined(vdisk): >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1374, in is_vdisk_defined >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server attrs = self.get_vdisk_attributes(vdisk_name) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1369, in get_vdisk_attributes >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server attrs = self.ssh.lsvdisk(vdisk) >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 467, in lsvdisk >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server with_header=False)[0] >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2079, in __getitem__ >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server return self.result[key] >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server IndexError: list index out of range >2018-08-29 05:33:16.654 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:19.304 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsiogrp -delim !: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsiogrp -delim ! >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.305 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsiogrp -delim ! >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.306 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsiogrp -delim !: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsiogrp -delim ! >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.307 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] CLI Exception output: > command: ['svcinfo', 'lsiogrp', '-delim', '!'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsiogrp -delim ! >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.315 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c2fc6684-0e04-46ae-9692-c8875bcaf31f) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:15Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:15Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54b034d0>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:15Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:15Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(765f9ae2-26db-4094-8cbc-2544c42b56a9),volume_id=765f9ae2-26db-4094-8cbc-2544c42b56a9,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab54b034d0>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9', 'type': 'raw', 'volume_id': '765f9ae2-26db-4094-8cbc-2544c42b56a9'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:15Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:15Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'reservations': [u'51729c74-3bb4-48df-93f9-2e9486ccb440', u'ffd21b3a-4902-4449-bfed-6a1a61e25c07'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a3118bb210a44f38854f389bfb790257', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'reservations': [u'51729c74-3bb4-48df-93f9-2e9486ccb440', u'ffd21b3a-4902-4449-bfed-6a1a61e25c07'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a3118bb210a44f38854f389bfb790257', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515', u'display_description': None, u'cluster_name': None, u'name_id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'volume_admin_metadata': [], u'project_id': u'a3118bb210a44f38854f389bfb790257', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9', u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'bootable': False, u'created_at': u'2018-08-29T05:33:14.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'group_id': None}, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab54b034d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(765f9ae2-26db-4094-8cbc-2544c42b56a9),volume_id=765f9ae2-26db-4094-8cbc-2544c42b56a9,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:15Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:15Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54b034d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:15Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:15Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsiogrp', '-delim', '!'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager ret = self.driver.create_volume(volume) >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager resp = self.ssh.lsiogrp() >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager command: ['svcinfo', 'lsiogrp', '-delim', '!'] >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager stdout: >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager . >2018-08-29 05:33:19.315 257256 ERROR cinder.volume.manager >2018-08-29 05:33:19.321 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c2fc6684-0e04-46ae-9692-c8875bcaf31f) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.324 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2757470c-9fe1-49f8-b639-090ca5ddf878) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.327 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (121a8d22-62cf-4de5-8477-8c1a4a5eee4d) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.362 257256 INFO cinder.rpc [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-29 05:33:19.411 257256 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.411 257256 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.411 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-765f9ae2-26db-4094-8cbc-2544c42b56a9": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.412 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.412 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-765f9ae2-26db-4094-8cbc-2544c42b56a9": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.414 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bbcb8769-61ca-4461-9ec9-ad7925e5f81e) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.417 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2634ac5a-09ee-4277-b72c-c573d11abaaf) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.419 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Flow 'volume_create_manager' (7d367a82-d89d-4935-b194-5597f861c877) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsiogrp', '-delim', '!'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server ret = self.driver.create_volume(volume) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server resp = self.ssh.lsiogrp() >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server command: ['svcinfo', 'lsiogrp', '-delim', '!'] >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server stdout: >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server . >2018-08-29 05:33:19.420 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:19.698 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.700 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.700 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.701 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] CLI Exception output: > command: ['svcinfo', 'lsfabric', '-delim', '!', '-wwpn', u'2100000e1ee8abd6'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.: ProcessExecutionError: Unexpected error while running command. >Command: svcinfo lsfabric -delim ! -wwpn 2100000e1ee8abd6 >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC5732E The command cannot be initiated because it was not run on the configuration node.\n' >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Terminate volume connection failed: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsfabric', '-delim', '!', '-wwpn', u'2100000e1ee8abd6'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >.: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsfabric', '-delim', '!', '-wwpn', u'2100000e1ee8abd6'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager force=force) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 261, in terminate_connection >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager return _do_terminate_connection_locked() >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager return f(*args, **kwargs) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 260, in _do_terminate_connection_locked >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager **kwargs) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 286, in _do_terminate_connection >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager connector, volume_name=vol_name) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 896, in get_host_from_connector >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager resp = self.ssh.lsfabric(wwpn=wwpn) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 275, in lsfabric >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager command: ['svcinfo', 'lsfabric', '-delim', '!', '-wwpn', u'2100000e1ee8abd6'] >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager stdout: >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager . >2018-08-29 05:33:19.703 257031 ERROR cinder.volume.manager >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server [req-a1e63c79-bba6-414f-9fd0-175849c2dc6c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svcinfo', 'lsfabric', '-delim', '!', '-wwpn', u'2100000e1ee8abd6'] > stdout: > stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >. >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server command: ['svcinfo', 'lsfabric', '-delim', '!', '-wwpn', u'2100000e1ee8abd6'] >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server stdout: >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server stderr: CMMVC5732E The command cannot be initiated because it was not run on the configuration node. >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server . >2018-08-29 05:33:19.705 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:19.818 257031 INFO cinder.volume.flows.manager.create_volume [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Volume 765f9ae2-26db-4094-8cbc-2544c42b56a9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9'} >2018-08-29 05:33:19.825 257031 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.825 257031 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.825 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.826 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.826 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.832 257031 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b31872e-dcb3-4d91-861e-63742f162650) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:19Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54740f90>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:19Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(765f9ae2-26db-4094-8cbc-2544c42b56a9),volume_id=765f9ae2-26db-4094-8cbc-2544c42b56a9,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab54740f90>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9', 'type': 'raw', 'volume_id': '765f9ae2-26db-4094-8cbc-2544c42b56a9'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:19Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'reservations': [u'51729c74-3bb4-48df-93f9-2e9486ccb440', u'ffd21b3a-4902-4449-bfed-6a1a61e25c07'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a3118bb210a44f38854f389bfb790257', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'reservations': [u'51729c74-3bb4-48df-93f9-2e9486ccb440', u'ffd21b3a-4902-4449-bfed-6a1a61e25c07'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a3118bb210a44f38854f389bfb790257', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515', u'display_description': None, u'cluster_name': None, u'name_id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'volume_admin_metadata': [], u'project_id': u'a3118bb210a44f38854f389bfb790257', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9', u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'bootable': False, u'created_at': u'2018-08-29T05:33:14.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'group_id': None}, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 2, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool', u'hostgroup@svc1#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool', u'hostgroup@svc1#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab54740f90>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(765f9ae2-26db-4094-8cbc-2544c42b56a9),volume_id=765f9ae2-26db-4094-8cbc-2544c42b56a9,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:19Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54740f90>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:19Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager ret = self.driver.create_volume(volume) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager resp = self.ssh.lsiogrp() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager command) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager check_exit_code, attempts) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager LOG.error("Error running SSH command: %s", command) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager with sshpool.item() as ssh: >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager return self.gen.next() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager obj = self.get() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager self.current_size -= 1 >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager new_conn = self.create() >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.832 257031 ERROR cinder.volume.manager >2018-08-29 05:33:19.839 257031 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b31872e-dcb3-4d91-861e-63742f162650) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.842 257031 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1667de44-14cb-4dae-af9f-5ffcbfbb3967) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.845 257031 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (995e0012-81cb-43e2-8840-05dda72256d6) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.914 257031 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.915 257031 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.915 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-765f9ae2-26db-4094-8cbc-2544c42b56a9": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.915 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.916 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-765f9ae2-26db-4094-8cbc-2544c42b56a9": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.917 257031 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4028702a-e90d-4547-8ea4-a7bd4c337832) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.920 257031 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0a7608fa-8bae-4813-8af6-b20f6b3d8b84) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:19.922 257031 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Flow 'volume_create_manager' (bf54de27-0648-4ce8-b5c5-1a24428520fb) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Exception during message handling: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server ret = self.driver.create_volume(volume) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server resp = self.ssh.lsiogrp() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server return self._ssh(ssh_cmd) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server command) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server check_exit_code, attempts) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server with sshpool.item() as ssh: >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server return self.gen.next() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server obj = self.get() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server self.current_size -= 1 >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server new_conn = self.create() >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server raise paramiko.SSHException(msg) >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:19.923 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:20.330 257256 INFO cinder.volume.flows.manager.create_volume [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Volume 765f9ae2-26db-4094-8cbc-2544c42b56a9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9'} >2018-08-29 05:33:20.338 257256 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.339 257256 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.339 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.340 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.340 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.347 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7a3be152-5c68-4f02-8481-b3c557745383) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:20Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:20Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5402b050>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:20Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:20Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(765f9ae2-26db-4094-8cbc-2544c42b56a9),volume_id=765f9ae2-26db-4094-8cbc-2544c42b56a9,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab5402b050>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9', 'type': 'raw', 'volume_id': '765f9ae2-26db-4094-8cbc-2544c42b56a9'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:20Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:20Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'reservations': [u'51729c74-3bb4-48df-93f9-2e9486ccb440', u'ffd21b3a-4902-4449-bfed-6a1a61e25c07'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a3118bb210a44f38854f389bfb790257', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'reservations': [u'51729c74-3bb4-48df-93f9-2e9486ccb440', u'ffd21b3a-4902-4449-bfed-6a1a61e25c07'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a3118bb210a44f38854f389bfb790257', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'size': 1, u'display_name': u'tempest-VolumesMetadataTest-Volume-317551515', u'display_description': None, u'cluster_name': None, u'name_id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'volume_admin_metadata': [], u'project_id': u'a3118bb210a44f38854f389bfb790257', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-765f9ae2-26db-4094-8cbc-2544c42b56a9', u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'bootable': False, u'created_at': u'2018-08-29T05:33:14.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'765f9ae2-26db-4094-8cbc-2544c42b56a9', u'group_id': None}, u'user_id': u'700da2c9b0f749668c1e1e0a17352038', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 3, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool', u'hostgroup@svc1#NonDR_DS8KBH00_Pool', u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool', u'hostgroup@svc1#NonDR_DS8KBH00_Pool', u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab5402b050>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(765f9ae2-26db-4094-8cbc-2544c42b56a9),volume_id=765f9ae2-26db-4094-8cbc-2544c42b56a9,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:20Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:20Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5402b050>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-317551515',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=765f9ae2-26db-4094-8cbc-2544c42b56a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a3118bb210a44f38854f389bfb790257',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:20Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:20Z,user_id='700da2c9b0f749668c1e1e0a17352038',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager ret = self.driver.create_volume(volume) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager resp = self.ssh.lsiogrp() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager command) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager check_exit_code, attempts) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager LOG.error("Error running SSH command: %s", command) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager with sshpool.item() as ssh: >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager return self.gen.next() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager obj = self.get() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager self.current_size -= 1 >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager new_conn = self.create() >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.347 257256 ERROR cinder.volume.manager >2018-08-29 05:33:20.355 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7a3be152-5c68-4f02-8481-b3c557745383) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:20.358 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (38470776-5c18-43f2-8b2c-6a49697d3ccb) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:20.361 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e323afec-eb57-4ae4-bc77-b760c07d1471) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:20.410 257256 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.410 257256 ERROR cinder.ssh_utils [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.411 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-765f9ae2-26db-4094-8cbc-2544c42b56a9": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.411 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.411 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-765f9ae2-26db-4094-8cbc-2544c42b56a9": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.413 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (845e822a-0f68-43f2-962b-068e1d9ecf6b) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:20.416 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d572454a-4949-4059-a547-6129a0bb3515) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:20.419 257256 WARNING cinder.volume.manager [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Flow 'volume_create_manager' (7ad46c8a-f017-41c1-9cd9-3401e4d813aa) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server [req-5356501a-a27b-4cb2-ac3f-8bc1ce354587 700da2c9b0f749668c1e1e0a17352038 a3118bb210a44f38854f389bfb790257 - default default] Exception during message handling: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server ret = self.driver.create_volume(volume) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server resp = self.ssh.lsiogrp() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server return self._ssh(ssh_cmd) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server command) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server check_exit_code, attempts) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server with sshpool.item() as ssh: >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server return self.gen.next() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server obj = self.get() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server self.current_size -= 1 >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server new_conn = self.create() >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server raise paramiko.SSHException(msg) >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:20.420 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:21.014 257031 ERROR cinder.ssh_utils [req-02c3ac45-9d19-47da-bfcb-50340cf77af0 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:21.014 257031 ERROR cinder.ssh_utils [req-02c3ac45-9d19-47da-bfcb-50340cf77af0 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:21.014 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-02c3ac45-9d19-47da-bfcb-50340cf77af0 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-9138881e-ce5f-4742-9fd6-1b5f57731404": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:21.015 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-02c3ac45-9d19-47da-bfcb-50340cf77af0 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:21.015 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-02c3ac45-9d19-47da-bfcb-50340cf77af0 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lsvdisk -bytes -delim ! "volume-9138881e-ce5f-4742-9fd6-1b5f57731404": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server [req-02c3ac45-9d19-47da-bfcb-50340cf77af0 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Exception during message handling: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 793, in delete_volume >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server new_status) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 775, in delete_volume >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2556, in delete_volume >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server self._helpers.delete_vdisk(volume['name'], False) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1852, in delete_vdisk >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server if not self.is_vdisk_defined(vdisk): >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1374, in is_vdisk_defined >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server attrs = self.get_vdisk_attributes(vdisk_name) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1369, in get_vdisk_attributes >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server attrs = self.ssh.lsvdisk(vdisk) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 464, in lsvdisk >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server out, err = self._ssh(ssh_cmd, check_exit_code=False) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server command) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server check_exit_code, attempts) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server with sshpool.item() as ssh: >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server return self.gen.next() >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server obj = self.get() >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server self.current_size -= 1 >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server new_conn = self.create() >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server raise paramiko.SSHException(msg) >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 05:33:21.049 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:33:23.370 257031 WARNING oslo_config.cfg [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-29 05:33:24.051 257031 INFO cinder.volume.flows.manager.create_volume [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Volume fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab540b5990>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:33:25.212 257031 INFO cinder.image.image_utils [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Image download 12.00 MB at 10.37 MB/s >2018-08-29 05:33:26.864 257256 INFO cinder.volume.flows.manager.create_volume [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Volume 0a47c415-e3a0-4837-9a9c-d962cd795ced: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0a47c415-e3a0-4837-9a9c-d962cd795ced'} >2018-08-29 05:33:44.470 257031 INFO cinder.volume.flows.manager.create_volume [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Volume 1788f762-a465-4328-b057-af9cce4bfb9c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1788f762-a465-4328-b057-af9cce4bfb9c'} >2018-08-29 05:33:45.309 257031 INFO cinder.volume.flows.manager.create_volume [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Volume f69793bc-b3aa-4cc9-9fac-337c49ec6214: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f69793bc-b3aa-4cc9-9fac-337c49ec6214'} >2018-08-29 05:33:56.309 257031 ERROR cinder.ssh_utils [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:33:56.311 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.312 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.312 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.325 257031 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (95af2e70-7651-489e-850a-6bddefc5d67c) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:23Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab544fd990>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:23Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b),volume_id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab544fd990>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': 'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab540b5990>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:23Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'availability_zone': u'nova', u'reservations': [u'1ce7a74f-bdfa-40d1-b032-dc433bfdb4f1', u'40d86ec2-6a84-46eb-a078-f9163dff7a3f'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'fbed6f2bbe6f4c93b83ad6ddd320307a', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-1198544035'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'availability_zone': u'nova', u'reservations': [u'1ce7a74f-bdfa-40d1-b032-dc433bfdb4f1', u'40d86ec2-6a84-46eb-a078-f9163dff7a3f'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'fbed6f2bbe6f4c93b83ad6ddd320307a', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-1198544035'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-1198544035', u'display_description': None, u'cluster_name': None, u'name_id': u'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'volume_admin_metadata': [], u'project_id': u'fbed6f2bbe6f4c93b83ad6ddd320307a', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'bootable': False, u'created_at': u'2018-08-29T05:33:22.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'group_id': None}, u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab544fd990>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b),volume_id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:23Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab544fd990>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:23Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager image_service) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager image_service >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager model_update = self.driver.create_volume(volume) or {} >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager resp = self.ssh.lsiogrp() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager command) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager check_exit_code, attempts) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager LOG.error("Error running SSH command: %s", command) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager with sshpool.item() as ssh: >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager return self.gen.next() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager obj = self.get() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager conn = super(SSHPool, self).get() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager created = self.create() >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.325 257031 ERROR cinder.volume.manager >2018-08-29 05:33:56.332 257031 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (95af2e70-7651-489e-850a-6bddefc5d67c) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:56.335 257031 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f802b0cb-81be-43d2-b7bc-3cd6b0a86e67) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:56.338 257031 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (39c0a59a-297c-47b4-bcb9-30b6f66df79a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:56.869 257256 ERROR cinder.ssh_utils [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:33:56.870 257256 ERROR cinder.ssh_utils [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.870 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.871 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.871 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.877 257256 WARNING cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (120f5f0e-b7da-4f3e-a65e-804a6fc1cf13) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1980271188',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=0a47c415-e3a0-4837-9a9c-d962cd795ced,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0fb42d7db7c7401b870633fd64557b25',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:26Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:26Z,user_id='3fc7394280fb45ed90832f6dad85bf2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5433e1d0>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1980271188',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=0a47c415-e3a0-4837-9a9c-d962cd795ced,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0fb42d7db7c7401b870633fd64557b25',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:26Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:26Z,user_id='3fc7394280fb45ed90832f6dad85bf2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(0a47c415-e3a0-4837-9a9c-d962cd795ced),volume_id=0a47c415-e3a0-4837-9a9c-d962cd795ced,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab5433e1d0>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0a47c415-e3a0-4837-9a9c-d962cd795ced', 'type': 'raw', 'volume_id': '0a47c415-e3a0-4837-9a9c-d962cd795ced'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1980271188',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=0a47c415-e3a0-4837-9a9c-d962cd795ced,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0fb42d7db7c7401b870633fd64557b25',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:26Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:26Z,user_id='3fc7394280fb45ed90832f6dad85bf2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'3fc7394280fb45ed90832f6dad85bf2e', u'availability_zone': u'nova', u'reservations': [u'41510295-5420-4589-bb54-3887f886d9c9', u'c6958d59-7c9a-4719-8efa-572605d0d1a6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'0fb42d7db7c7401b870633fd64557b25', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesSnapshotTestJSON-Volume-1980271188'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'3fc7394280fb45ed90832f6dad85bf2e', u'availability_zone': u'nova', u'reservations': [u'41510295-5420-4589-bb54-3887f886d9c9', u'c6958d59-7c9a-4719-8efa-572605d0d1a6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'0fb42d7db7c7401b870633fd64557b25', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesSnapshotTestJSON-Volume-1980271188'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'0a47c415-e3a0-4837-9a9c-d962cd795ced', u'size': 1, u'display_name': u'tempest-VolumesSnapshotTestJSON-Volume-1980271188', u'display_description': None, u'cluster_name': None, u'name_id': u'0a47c415-e3a0-4837-9a9c-d962cd795ced', u'volume_admin_metadata': [], u'project_id': u'0fb42d7db7c7401b870633fd64557b25', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-0a47c415-e3a0-4837-9a9c-d962cd795ced', u'user_id': u'3fc7394280fb45ed90832f6dad85bf2e', u'bootable': False, u'created_at': u'2018-08-29T05:33:26.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'0a47c415-e3a0-4837-9a9c-d962cd795ced', u'group_id': None}, u'user_id': u'3fc7394280fb45ed90832f6dad85bf2e', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab5433e1d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(0a47c415-e3a0-4837-9a9c-d962cd795ced),volume_id=0a47c415-e3a0-4837-9a9c-d962cd795ced,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1980271188',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=0a47c415-e3a0-4837-9a9c-d962cd795ced,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0fb42d7db7c7401b870633fd64557b25',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:26Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:26Z,user_id='3fc7394280fb45ed90832f6dad85bf2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5433e1d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1980271188',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=0a47c415-e3a0-4837-9a9c-d962cd795ced,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0fb42d7db7c7401b870633fd64557b25',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:26Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:26Z,user_id='3fc7394280fb45ed90832f6dad85bf2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager ret = self.driver.create_volume(volume) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager resp = self.ssh.lsiogrp() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager command) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager check_exit_code, attempts) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager LOG.error("Error running SSH command: %s", command) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager with sshpool.item() as ssh: >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager return self.gen.next() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager obj = self.get() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager self.current_size -= 1 >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager new_conn = self.create() >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-29 05:33:56.877 257256 ERROR cinder.volume.manager >2018-08-29 05:33:56.884 257256 WARNING cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (120f5f0e-b7da-4f3e-a65e-804a6fc1cf13) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:56.887 257256 WARNING cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (16fd4bc3-093a-414a-bc07-708127b48527) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:56.891 257256 WARNING cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4f9f2385-0f12-4f42-9e2c-ae0484d6d017) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:33:57.312 257031 INFO cinder.volume.flows.manager.create_volume [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Volume 0a47c415-e3a0-4837-9a9c-d962cd795ced: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0a47c415-e3a0-4837-9a9c-d962cd795ced'} >2018-08-29 05:33:57.712 257256 INFO cinder.volume.flows.manager.create_volume [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Volume fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54977fd0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:33:57.778 257256 INFO cinder.volume.flows.manager.create_volume [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Volume 29fa690f-9ad4-49ce-9856-5a8a8c727431: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-29fa690f-9ad4-49ce-9856-5a8a8c727431', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54a847d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:33:58.875 257256 INFO cinder.image.image_utils [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Image download 12.00 MB at 10.35 MB/s >2018-08-29 05:34:02.965 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:34:03.423 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:34:03.439 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:34:03.510 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:34:03.924 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Tried to delete non-existent vdisk volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b. >2018-08-29 05:34:03.926 257031 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51542ded-3922-47e1-911f-4b1acb4503d4) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:03.930 257031 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2bdf0744-5b66-497d-868c-4829415b52cb) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:03.932 257031 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Flow 'volume_create_manager' (151fdebe-2e74-433b-bfc3-2b043d6f5a08) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Exception during message handling: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 685, in _create_from_image_download >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server model_update = self.driver.create_volume(volume) or {} >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server resp = self.ssh.lsiogrp() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server return self._ssh(ssh_cmd) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server command) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server check_exit_code, attempts) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server with sshpool.item() as ssh: >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server return self.gen.next() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server obj = self.get() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server conn = super(SSHPool, self).get() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server created = self.create() >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server raise paramiko.SSHException(msg) >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:03.933 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:34:03.958 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:34:04.042 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:34:04.603 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Tried to delete non-existent vdisk volume-0a47c415-e3a0-4837-9a9c-d962cd795ced. >2018-08-29 05:34:04.607 257256 WARNING cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f9b914d7-f41c-49d8-9a60-3eb69efadb53) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:04.610 257256 WARNING cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (57c7b416-5f75-4810-a6cc-99301bc8ba55) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:04.613 257256 WARNING cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Flow 'volume_create_manager' (81ecdaae-7800-4207-a7a7-cc9c38149dd0) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Exception during message handling: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server ret = self.driver.create_volume(volume) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server resp = self.ssh.lsiogrp() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server return self._ssh(ssh_cmd) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server command) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server check_exit_code, attempts) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server with sshpool.item() as ssh: >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server return self.gen.next() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server obj = self.get() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server self.current_size -= 1 >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server new_conn = self.create() >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server raise paramiko.SSHException(msg) >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:04.614 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:34:04.993 257031 INFO cinder.volume.flows.manager.create_volume [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Volume volume-0a47c415-e3a0-4837-9a9c-d962cd795ced (0a47c415-e3a0-4837-9a9c-d962cd795ced): created successfully >2018-08-29 05:34:04.997 257031 INFO cinder.volume.manager [req-3be963a4-7034-41b6-bb06-35542d7513f9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Created volume successfully. >2018-08-29 05:34:05.594 257031 ERROR cinder.ssh_utils [req-f4eb92c2-e8ca-4197-9f5b-d30a95b622a8 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:34:05.595 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f4eb92c2-e8ca-4197-9f5b-d30a95b622a8 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Error running SSH command: svcinfo lsfabric -delim ! -wwpn 2100000e1ec25b5a: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:05.595 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f4eb92c2-e8ca-4197-9f5b-d30a95b622a8 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:05.596 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f4eb92c2-e8ca-4197-9f5b-d30a95b622a8 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Error running SSH command: svcinfo lsfabric -delim ! -wwpn 2100000e1ec25b5a: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager [req-f4eb92c2-e8ca-4197-9f5b-d30a95b622a8 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Terminate volume connection failed: Error connecting via ssh: timed out: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager force=force) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 104, in decorator >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager conn_info = terminate_connection(self, *args, **kwargs) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 261, in terminate_connection >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager return _do_terminate_connection_locked() >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager return f(*args, **kwargs) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 260, in _do_terminate_connection_locked >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager **kwargs) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 286, in _do_terminate_connection >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager connector, volume_name=vol_name) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 896, in get_host_from_connector >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager resp = self.ssh.lsfabric(wwpn=wwpn) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 275, in lsfabric >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager command) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager check_exit_code, attempts) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager with sshpool.item() as ssh: >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager return self.gen.next() >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager obj = self.get() >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager conn = super(SSHPool, self).get() >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager created = self.create() >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:05.598 257031 ERROR cinder.volume.manager >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server [req-f4eb92c2-e8ca-4197-9f5b-d30a95b622a8 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Error connecting via ssh: timed out >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Error connecting via ssh: timed out >2018-08-29 05:34:05.600 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:34:09.486 257031 INFO cinder.volume.manager [req-f06dee98-5e34-40e4-8aef-2756703cd7b7 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:34:13.342 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:34:13.344 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.345 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.345 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.345 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task command) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.346 257256 ERROR oslo_service.periodic_task >2018-08-29 05:34:13.398 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:34:13.399 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.400 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.400 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task command) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task conn = super(SSHPool, self).get() >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task created = self.create() >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:13.400 257031 ERROR oslo_service.periodic_task >2018-08-29 05:34:14.479 257031 ERROR cinder.ssh_utils [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:34:14.480 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:14.481 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:14.481 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:14.489 257031 WARNING cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7a84e392-319e-4dd1-a289-d4e257014f5d) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1222680326',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=1788f762-a465-4328-b057-af9cce4bfb9c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d2fabfc7c9f5498c9c88525d8baf9b5f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:44Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:44Z,user_id='668e74d2df7245f3a541e7d5f2b9f5ad',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54450c90>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1222680326',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=1788f762-a465-4328-b057-af9cce4bfb9c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d2fabfc7c9f5498c9c88525d8baf9b5f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:44Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:44Z,user_id='668e74d2df7245f3a541e7d5f2b9f5ad',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(1788f762-a465-4328-b057-af9cce4bfb9c),volume_id=1788f762-a465-4328-b057-af9cce4bfb9c,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab54450c90>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1788f762-a465-4328-b057-af9cce4bfb9c', 'type': 'raw', 'volume_id': '1788f762-a465-4328-b057-af9cce4bfb9c'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1222680326',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=1788f762-a465-4328-b057-af9cce4bfb9c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d2fabfc7c9f5498c9c88525d8baf9b5f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:44Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:44Z,user_id='668e74d2df7245f3a541e7d5f2b9f5ad',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'668e74d2df7245f3a541e7d5f2b9f5ad', u'availability_zone': u'nova', u'reservations': [u'eebb429e-87f1-4339-a1d4-6154b1042ac2', u'885a941b-93df-458f-a8ba-67d9288dce75'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'd2fabfc7c9f5498c9c88525d8baf9b5f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-AttachVolumeTestJSON-volume-1222680326'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'668e74d2df7245f3a541e7d5f2b9f5ad', u'availability_zone': u'nova', u'reservations': [u'eebb429e-87f1-4339-a1d4-6154b1042ac2', u'885a941b-93df-458f-a8ba-67d9288dce75'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'd2fabfc7c9f5498c9c88525d8baf9b5f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-AttachVolumeTestJSON-volume-1222680326'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'1788f762-a465-4328-b057-af9cce4bfb9c', u'size': 1, u'display_name': u'tempest-AttachVolumeTestJSON-volume-1222680326', u'display_description': None, u'cluster_name': None, u'name_id': u'1788f762-a465-4328-b057-af9cce4bfb9c', u'volume_admin_metadata': [], u'project_id': u'd2fabfc7c9f5498c9c88525d8baf9b5f', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-1788f762-a465-4328-b057-af9cce4bfb9c', u'user_id': u'668e74d2df7245f3a541e7d5f2b9f5ad', u'bootable': False, u'created_at': u'2018-08-29T05:33:43.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'1788f762-a465-4328-b057-af9cce4bfb9c', u'group_id': None}, u'user_id': u'668e74d2df7245f3a541e7d5f2b9f5ad', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab54450c90>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(1788f762-a465-4328-b057-af9cce4bfb9c),volume_id=1788f762-a465-4328-b057-af9cce4bfb9c,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1222680326',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=1788f762-a465-4328-b057-af9cce4bfb9c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d2fabfc7c9f5498c9c88525d8baf9b5f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:44Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:44Z,user_id='668e74d2df7245f3a541e7d5f2b9f5ad',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54450c90>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1222680326',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=1788f762-a465-4328-b057-af9cce4bfb9c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d2fabfc7c9f5498c9c88525d8baf9b5f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:44Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:44Z,user_id='668e74d2df7245f3a541e7d5f2b9f5ad',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager ret = self.driver.create_volume(volume) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager resp = self.ssh.lsiogrp() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager command) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager check_exit_code, attempts) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager with sshpool.item() as ssh: >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager return self.gen.next() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager obj = self.get() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager conn = super(SSHPool, self).get() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager created = self.create() >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:14.489 257031 ERROR cinder.volume.manager >2018-08-29 05:34:14.497 257031 WARNING cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7a84e392-319e-4dd1-a289-d4e257014f5d) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:14.500 257031 WARNING cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8a0dc84-ea31-40f1-9150-c22436757510) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:14.504 257031 WARNING cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (18697864-61f1-4404-8405-bb9494db7916) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:14.562 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:34:14.637 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:34:14.779 257031 INFO cinder.volume.flows.manager.create_volume [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Volume 1788f762-a465-4328-b057-af9cce4bfb9c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1788f762-a465-4328-b057-af9cce4bfb9c'} >2018-08-29 05:34:14.792 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:34:14.859 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:34:15.314 257031 ERROR cinder.ssh_utils [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 05:34:15.315 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:15.316 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:15.316 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Error running SSH command: svcinfo lsiogrp -delim !: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:15.322 257031 WARNING cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4e8340b4-23d4-4b26-9ff8-b584ef29816b) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-614463413',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=f69793bc-b3aa-4cc9-9fac-337c49ec6214,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a397d15d5b51485bafeba5ca087d7482',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:45Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:45Z,user_id='4c9e0a6f2cf54d2cb246178c892e6bf0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab547b7e90>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-614463413',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=f69793bc-b3aa-4cc9-9fac-337c49ec6214,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a397d15d5b51485bafeba5ca087d7482',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:45Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:45Z,user_id='4c9e0a6f2cf54d2cb246178c892e6bf0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(f69793bc-b3aa-4cc9-9fac-337c49ec6214),volume_id=f69793bc-b3aa-4cc9-9fac-337c49ec6214,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab547b7e90>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f69793bc-b3aa-4cc9-9fac-337c49ec6214', 'type': 'raw', 'volume_id': 'f69793bc-b3aa-4cc9-9fac-337c49ec6214'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-614463413',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=f69793bc-b3aa-4cc9-9fac-337c49ec6214,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a397d15d5b51485bafeba5ca087d7482',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:45Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:45Z,user_id='4c9e0a6f2cf54d2cb246178c892e6bf0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'4c9e0a6f2cf54d2cb246178c892e6bf0', u'availability_zone': u'nova', u'reservations': [u'5c4aa9ad-fa93-4def-82ef-afa79e7f0210', u'2ce1a79a-22dc-4a30-9784-a46afd460415'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a397d15d5b51485bafeba5ca087d7482', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-AbsoluteLimitsTests-Volume-614463413'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'4c9e0a6f2cf54d2cb246178c892e6bf0', u'availability_zone': u'nova', u'reservations': [u'5c4aa9ad-fa93-4def-82ef-afa79e7f0210', u'2ce1a79a-22dc-4a30-9784-a46afd460415'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a397d15d5b51485bafeba5ca087d7482', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-AbsoluteLimitsTests-Volume-614463413'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'f69793bc-b3aa-4cc9-9fac-337c49ec6214', u'size': 1, u'display_name': u'tempest-AbsoluteLimitsTests-Volume-614463413', u'display_description': None, u'cluster_name': None, u'name_id': u'f69793bc-b3aa-4cc9-9fac-337c49ec6214', u'volume_admin_metadata': [], u'project_id': u'a397d15d5b51485bafeba5ca087d7482', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-f69793bc-b3aa-4cc9-9fac-337c49ec6214', u'user_id': u'4c9e0a6f2cf54d2cb246178c892e6bf0', u'bootable': False, u'created_at': u'2018-08-29T05:33:44.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'f69793bc-b3aa-4cc9-9fac-337c49ec6214', u'group_id': None}, u'user_id': u'4c9e0a6f2cf54d2cb246178c892e6bf0', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab547b7e90>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(f69793bc-b3aa-4cc9-9fac-337c49ec6214),volume_id=f69793bc-b3aa-4cc9-9fac-337c49ec6214,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-614463413',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=f69793bc-b3aa-4cc9-9fac-337c49ec6214,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a397d15d5b51485bafeba5ca087d7482',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:45Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:45Z,user_id='4c9e0a6f2cf54d2cb246178c892e6bf0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab547b7e90>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-614463413',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=f69793bc-b3aa-4cc9-9fac-337c49ec6214,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a397d15d5b51485bafeba5ca087d7482',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:45Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-08-29T05:33:45Z,user_id='4c9e0a6f2cf54d2cb246178c892e6bf0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager ret = self.driver.create_volume(volume) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager resp = self.ssh.lsiogrp() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager return self._ssh(ssh_cmd) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager command) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager check_exit_code, attempts) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager self.force_reraise() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager with sshpool.item() as ssh: >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager return self.gen.next() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager obj = self.get() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager conn = super(SSHPool, self).get() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager created = self.create() >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager raise paramiko.SSHException(msg) >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:15.322 257031 ERROR cinder.volume.manager >2018-08-29 05:34:15.329 257031 WARNING cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4e8340b4-23d4-4b26-9ff8-b584ef29816b) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:15.332 257031 WARNING cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa2a8516-b506-443e-a1f0-9ff8ec8f1260) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:15.335 257031 WARNING cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ab7f91bf-5f5a-414f-8d92-6d0428dbf16a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:15.373 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:34:15.459 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:34:15.606 257256 INFO cinder.volume.flows.manager.create_volume [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Volume f69793bc-b3aa-4cc9-9fac-337c49ec6214: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f69793bc-b3aa-4cc9-9fac-337c49ec6214'} >2018-08-29 05:34:37.713 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Tried to delete non-existent vdisk volume-1788f762-a465-4328-b057-af9cce4bfb9c. >2018-08-29 05:34:37.719 257031 WARNING cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a8fc6195-d98a-4882-b882-07b01c14c52f) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:37.722 257031 WARNING cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (618445c2-f06f-49bb-8416-f8f535f10f0a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:37.725 257031 WARNING cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Flow 'volume_create_manager' (49066d63-7eb3-4f68-ab8b-354391c7390e) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Exception during message handling: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server ret = self.driver.create_volume(volume) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server resp = self.ssh.lsiogrp() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server return self._ssh(ssh_cmd) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server command) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server check_exit_code, attempts) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server with sshpool.item() as ssh: >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server return self.gen.next() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server obj = self.get() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server conn = super(SSHPool, self).get() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server created = self.create() >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server raise paramiko.SSHException(msg) >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:37.726 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:34:37.759 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Tried to delete non-existent vdisk volume-f69793bc-b3aa-4cc9-9fac-337c49ec6214. >2018-08-29 05:34:37.761 257031 WARNING cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7c510a7e-9d15-4777-9844-0fcdbff83d54) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:37.765 257031 WARNING cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ba82d409-4d23-4bc6-a5a5-2c681845b24b) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:37.768 257031 WARNING cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Flow 'volume_create_manager' (ecd5da7e-d02c-4d95-a877-638e38956222) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Exception during message handling: SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server model_update = self._create_raw_volume(volume, **volume_spec) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server ret = self.driver.create_volume(volume) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2507, in create_volume >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server opts['iogrp'] = self._helpers.select_io_group(self._state, opts) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 766, in select_io_group >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server iog_vdc = self.get_vdisk_count_by_io_group() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 743, in get_vdisk_count_by_io_group >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server resp = self.ssh.lsiogrp() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 223, in lsiogrp >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server return self.run_ssh_info(ssh_cmd, with_header=True) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server raw = self._run_ssh(ssh_cmd) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server return self._ssh(ssh_cmd) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server command) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server check_exit_code, attempts) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server LOG.error("Error running SSH command: %s", command) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server with sshpool.item() as ssh: >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server return self.gen.next() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server obj = self.get() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 164, in get >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server conn = super(SSHPool, self).get() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 88, in get >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server created = self.create() >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server raise paramiko.SSHException(msg) >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server SSHException: Error connecting via ssh: timed out >2018-08-29 05:34:37.769 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:34:39.730 257031 INFO cinder.volume.flows.manager.create_volume [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Volume volume-1788f762-a465-4328-b057-af9cce4bfb9c (1788f762-a465-4328-b057-af9cce4bfb9c): created successfully >2018-08-29 05:34:39.736 257031 INFO cinder.volume.manager [req-73365c00-5d04-4db1-9355-c8fee84114a1 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Created volume successfully. >2018-08-29 05:34:40.062 257256 INFO cinder.volume.flows.manager.create_volume [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Volume volume-f69793bc-b3aa-4cc9-9fac-337c49ec6214 (f69793bc-b3aa-4cc9-9fac-337c49ec6214): created successfully >2018-08-29 05:34:40.067 257256 INFO cinder.volume.manager [req-68e3a50d-14c2-4867-b3ff-a0686922f545 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Created volume successfully. >2018-08-29 05:34:43.357 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:34:43.467 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:34:44.298 257256 INFO cinder.volume.manager [req-6c52d8c1-5963-4fc2-957a-96ee633c9754 4c9e0a6f2cf54d2cb246178c892e6bf0 a397d15d5b51485bafeba5ca087d7482 - default default] Deleted volume successfully. >2018-08-29 05:34:44.404 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] initialize_connection: Did not find a preferred node for volume volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b. >2018-08-29 05:34:46.093 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:34:47.259 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-0933a376-551d-4d5b-a59a-49a3ddc6866e 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] initialize_connection: Did not find a preferred node for volume volume-1788f762-a465-4328-b057-af9cce4bfb9c. >2018-08-29 05:34:47.658 257031 INFO cinder.volume.manager [req-0933a376-551d-4d5b-a59a-49a3ddc6866e 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Initialize volume connection completed successfully. >2018-08-29 05:34:49.033 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:34:49.322 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:34:49.334 257256 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a7c16b7e-bcc6-47f9-a622-555be325ac5f) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:56Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:56Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab548b1b50>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:56Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:56Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b),volume_id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab548b1b50>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': 'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54977fd0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:56Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:56Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'availability_zone': u'nova', u'reservations': [u'1ce7a74f-bdfa-40d1-b032-dc433bfdb4f1', u'40d86ec2-6a84-46eb-a078-f9163dff7a3f'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'fbed6f2bbe6f4c93b83ad6ddd320307a', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-1198544035'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'availability_zone': u'nova', u'reservations': [u'1ce7a74f-bdfa-40d1-b032-dc433bfdb4f1', u'40d86ec2-6a84-46eb-a078-f9163dff7a3f'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'fbed6f2bbe6f4c93b83ad6ddd320307a', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-1198544035'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-1198544035', u'display_description': None, u'cluster_name': None, u'name_id': u'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'volume_admin_metadata': [], u'project_id': u'fbed6f2bbe6f4c93b83ad6ddd320307a', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'bootable': False, u'created_at': u'2018-08-29T05:33:22.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b', u'group_id': None}, u'user_id': u'196daa0bca4b4e6e9f461e6d059bc482', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 2, u'backends': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool', u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool', u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab548b1b50>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b),volume_id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:56Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:56Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab548b1b50>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1198544035',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbed6f2bbe6f4c93b83ad6ddd320307a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:56Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:56Z,user_id='196daa0bca4b4e6e9f461e6d059bc482',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:34:49.334 257256 ERROR cinder.volume.manager >2018-08-29 05:34:49.341 257256 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a7c16b7e-bcc6-47f9-a622-555be325ac5f) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:49.344 257256 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (42386414-cb34-4cc1-bfe6-37773f29662a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:49.348 257256 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d60af848-4c3c-459d-a2d8-492c6d6df679) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:49.366 257256 ERROR cinder.volume.flows.manager.create_volume [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Volume fa6b0074-6b2e-44bf-b3a8-e89ef9a36a9b: create failed >2018-08-29 05:34:49.368 257256 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (640ab9f9-f8f3-4548-9e1a-8128e648c4f1) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:49.371 257256 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ae825d2-9319-4072-a220-5679f4f19036) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:34:49.373 257256 WARNING cinder.volume.manager [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Flow 'volume_create_manager' (42d7ded0-797c-45cd-acfc-b70d803f451b) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server [req-5c27a8dc-3735-459c-a61e-63a79d2b39fd 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:34:49.383 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:34:50.711 257256 INFO cinder.image.image_utils [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Image download 12.00 MB at 8.84 MB/s >2018-08-29 05:34:50.779 257031 INFO cinder.volume.flows.manager.create_volume [req-aee06e2c-0daa-4bfc-9430-1165266cf34a 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Volume 1306947e-1655-4823-aae8-fe61b16a0f3e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1306947e-1655-4823-aae8-fe61b16a0f3e'} >2018-08-29 05:34:51.831 257031 INFO cinder.volume.flows.manager.create_volume [req-aee06e2c-0daa-4bfc-9430-1165266cf34a 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Volume volume-1306947e-1655-4823-aae8-fe61b16a0f3e (1306947e-1655-4823-aae8-fe61b16a0f3e): created successfully >2018-08-29 05:34:51.838 257031 INFO cinder.volume.manager [req-aee06e2c-0daa-4bfc-9430-1165266cf34a 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Created volume successfully. >2018-08-29 05:34:52.143 257031 INFO cinder.volume.flows.manager.create_volume [req-62034326-4605-42c1-97e9-6a6183c70b6c fde22d94851d4636962866e20f639670 6cdb60e10db1434ab6c58db70646488a - default default] Volume 4e9388f2-5a53-4086-8a22-af08b3a78e92: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-4e9388f2-5a53-4086-8a22-af08b3a78e92'} >2018-08-29 05:34:53.960 257031 INFO cinder.volume.flows.manager.create_volume [req-62034326-4605-42c1-97e9-6a6183c70b6c fde22d94851d4636962866e20f639670 6cdb60e10db1434ab6c58db70646488a - default default] Volume volume-4e9388f2-5a53-4086-8a22-af08b3a78e92 (4e9388f2-5a53-4086-8a22-af08b3a78e92): created successfully >2018-08-29 05:34:53.966 257031 INFO cinder.volume.manager [req-62034326-4605-42c1-97e9-6a6183c70b6c fde22d94851d4636962866e20f639670 6cdb60e10db1434ab6c58db70646488a - default default] Created volume successfully. >2018-08-29 05:34:54.176 257031 INFO cinder.volume.manager [req-fe241a2d-0be6-4331-bfc8-382791f7d608 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Attaching volume 1788f762-a465-4328-b057-af9cce4bfb9c to instance ada7ff8c-e009-4004-b0a1-c610663544d4 at mountpoint /dev/vdb on host None. >2018-08-29 05:34:54.267 257031 INFO cinder.volume.manager [req-fe241a2d-0be6-4331-bfc8-382791f7d608 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Attach volume completed successfully. >2018-08-29 05:34:55.234 257031 INFO cinder.volume.flows.manager.create_volume [req-80c3de37-19dd-4c2d-923d-ead229d14e59 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Volume 3e874d12-f73d-45b6-910b-75979b6ec9a8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3e874d12-f73d-45b6-910b-75979b6ec9a8'} >2018-08-29 05:34:56.982 257031 INFO cinder.volume.flows.manager.create_volume [req-80c3de37-19dd-4c2d-923d-ead229d14e59 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Volume volume-3e874d12-f73d-45b6-910b-75979b6ec9a8 (3e874d12-f73d-45b6-910b-75979b6ec9a8): created successfully >2018-08-29 05:34:56.987 257031 INFO cinder.volume.manager [req-80c3de37-19dd-4c2d-923d-ead229d14e59 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Created volume successfully. >2018-08-29 05:34:57.749 257031 INFO cinder.volume.manager [req-8dc71e25-fb53-4692-8b49-b02c466e0033 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Create snapshot completed successfully >2018-08-29 05:34:58.601 257031 INFO cinder.volume.manager [req-7953609d-0d6c-49eb-b615-e6a7276b850f fde22d94851d4636962866e20f639670 6cdb60e10db1434ab6c58db70646488a - default default] Deleted volume successfully. >2018-08-29 05:35:02.481 257256 INFO cinder.volume.flows.manager.create_volume [req-df2169f6-13c7-4878-8397-a433808829f7 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] Volume d93ab77d-294d-4d92-91a8-af050bd96b86: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d93ab77d-294d-4d92-91a8-af050bd96b86'} >2018-08-29 05:35:02.619 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-58dc0db8-05af-44ec-ac61-d4edb79758b9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] initialize_connection: Did not find a preferred node for volume volume-d1428df6-61ea-42fe-840a-4e4c65833026. >2018-08-29 05:35:03.660 257256 INFO cinder.volume.flows.manager.create_volume [req-df2169f6-13c7-4878-8397-a433808829f7 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] Volume volume-d93ab77d-294d-4d92-91a8-af050bd96b86 (d93ab77d-294d-4d92-91a8-af050bd96b86): created successfully >2018-08-29 05:35:03.665 257256 INFO cinder.volume.manager [req-df2169f6-13c7-4878-8397-a433808829f7 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] Created volume successfully. >2018-08-29 05:35:04.216 257031 INFO cinder.volume.manager [req-58dc0db8-05af-44ec-ac61-d4edb79758b9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Initialize volume connection completed successfully. >2018-08-29 05:35:04.701 257256 INFO cinder.volume.flows.manager.create_volume [req-a272206e-220d-4322-bcee-39c408108281 decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] Volume 65079cf1-dbd8-4427-a4b0-cf318d7cc531: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-65079cf1-dbd8-4427-a4b0-cf318d7cc531'} >2018-08-29 05:35:05.681 257256 INFO cinder.volume.manager [req-cbbce2ff-162d-4b9c-b4f9-b2657b9256cd 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] Retype volume completed successfully. >2018-08-29 05:35:06.196 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-75f3bee9-93dc-41e6-89d8-3b54facfa196 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] initialize_connection: Did not find a preferred node for volume volume-3e874d12-f73d-45b6-910b-75979b6ec9a8. >2018-08-29 05:35:06.409 257256 INFO cinder.volume.flows.manager.create_volume [req-a272206e-220d-4322-bcee-39c408108281 decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] Volume volume-65079cf1-dbd8-4427-a4b0-cf318d7cc531 (65079cf1-dbd8-4427-a4b0-cf318d7cc531): created successfully >2018-08-29 05:35:06.413 257256 INFO cinder.volume.manager [req-a272206e-220d-4322-bcee-39c408108281 decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] Created volume successfully. >2018-08-29 05:35:06.653 257031 INFO cinder.volume.manager [req-75f3bee9-93dc-41e6-89d8-3b54facfa196 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Initialize volume connection completed successfully. >2018-08-29 05:35:07.775 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-67acb42f-3da4-45c5-a8b5-c737faa3634c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] initialize_connection: Did not find a preferred node for volume volume-35425500-d7ef-4ee8-bb56-d56034b7e40e. >2018-08-29 05:35:08.942 257031 INFO cinder.volume.manager [req-67acb42f-3da4-45c5-a8b5-c737faa3634c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Initialize volume connection completed successfully. >2018-08-29 05:35:09.437 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c9328047-1d42-4ea7-8d63-ee8857fc03bf decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] Unable to get remote copy information for volume volume-65079cf1-dbd8-4427-a4b0-cf318d7cc531 >2018-08-29 05:35:10.448 257256 INFO cinder.volume.manager [req-c9328047-1d42-4ea7-8d63-ee8857fc03bf decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] Extend volume completed successfully. >2018-08-29 05:35:10.816 257256 WARNING cinder.quota [req-5e4aa2a6-bfd8-412d-ade6-d217a8bee44d 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1033221649 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1033221649, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:35:10.818 257256 WARNING cinder.quota [req-5e4aa2a6-bfd8-412d-ade6-d217a8bee44d 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1033221649 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1033221649, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:35:11.367 257031 INFO cinder.volume.manager [req-a1b1738f-23e4-4462-9e2d-e9a8ad00c05d 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Attaching volume 3e874d12-f73d-45b6-910b-75979b6ec9a8 to instance ada7ff8c-e009-4004-b0a1-c610663544d4 at mountpoint /dev/vdc on host None. >2018-08-29 05:35:11.466 257031 INFO cinder.volume.manager [req-a1b1738f-23e4-4462-9e2d-e9a8ad00c05d 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Attach volume completed successfully. >2018-08-29 05:35:11.474 257256 INFO cinder.volume.manager [req-5e4aa2a6-bfd8-412d-ade6-d217a8bee44d 5ec6ff5384fe4bf3bbb28e65d4255f19 4213cb402c5d4ae89e61e0c21ac9111b - default default] Deleted volume successfully. >2018-08-29 05:35:12.689 257031 INFO cinder.volume.manager [req-58dc0db8-05af-44ec-ac61-d4edb79758b9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Terminate volume connection completed successfully. >2018-08-29 05:35:15.924 257256 INFO cinder.volume.manager [req-4e35bf5e-64c9-410f-afba-8e3d41ccb4fb decbcaf1d39b4752906169af82041bdf c0397f1585d643adb3c14c013140ea5f - default default] Deleted volume successfully. >2018-08-29 05:35:17.659 257031 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 0.63 sec >2018-08-29 05:35:19.242 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-67acb42f-3da4-45c5-a8b5-c737faa3634c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:35:19.922 257031 INFO cinder.volume.manager [req-67acb42f-3da4-45c5-a8b5-c737faa3634c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Terminate volume connection completed successfully. >2018-08-29 05:35:19.998 257031 INFO cinder.volume.manager [req-58dc0db8-05af-44ec-ac61-d4edb79758b9 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Deleted volume successfully. >2018-08-29 05:35:21.926 257031 INFO cinder.volume.flows.manager.create_volume [req-dd8700d1-e1ab-4850-8f01-6e24afb0a33a 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Volume b88d5487-2a05-44b7-97d1-a0887414375d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b88d5487-2a05-44b7-97d1-a0887414375d'} >2018-08-29 05:35:23.533 257256 INFO cinder.volume.flows.manager.create_volume [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Volume 56c77037-4f08-4104-8bfd-7bf9f7ede1e5: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-56c77037-4f08-4104-8bfd-7bf9f7ede1e5', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab540c9450>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:35:24.675 257031 INFO cinder.volume.flows.manager.create_volume [req-dd8700d1-e1ab-4850-8f01-6e24afb0a33a 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Volume volume-b88d5487-2a05-44b7-97d1-a0887414375d (b88d5487-2a05-44b7-97d1-a0887414375d): created successfully >2018-08-29 05:35:24.679 257031 INFO cinder.volume.manager [req-dd8700d1-e1ab-4850-8f01-6e24afb0a33a 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Created volume successfully. >2018-08-29 05:35:26.525 257031 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 2.82 sec >2018-08-29 05:35:26.577 257031 INFO cinder.volume.manager [req-afb426b4-49b2-4a20-a10b-7f8187f8cd2a 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Terminate volume connection completed successfully. >2018-08-29 05:35:26.745 257031 INFO cinder.volume.manager [req-323b7864-d8b2-47c5-8e3e-b87a54a92d39 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Detaching volume 1788f762-a465-4328-b057-af9cce4bfb9c from instance ada7ff8c-e009-4004-b0a1-c610663544d4. >2018-08-29 05:35:26.837 257031 INFO cinder.volume.manager [req-323b7864-d8b2-47c5-8e3e-b87a54a92d39 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Detach volume completed successfully. >2018-08-29 05:35:31.563 257031 INFO cinder.volume.manager [req-67acb42f-3da4-45c5-a8b5-c737faa3634c 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Deleted volume successfully. >2018-08-29 05:35:33.346 257031 INFO cinder.volume.manager [req-232873b0-49e4-4445-85c5-0ba4fa2eacda 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Delete snapshot completed successfully. >2018-08-29 05:35:33.904 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b7fe089a-cf87-4c23-973a-5b021ed435e5 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] initialize_connection: Did not find a preferred node for volume volume-0a47c415-e3a0-4837-9a9c-d962cd795ced. >2018-08-29 05:35:34.194 257031 INFO cinder.volume.manager [req-80afc445-0f06-4d60-979d-cdca7a14b5f1 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Create snapshot completed successfully >2018-08-29 05:35:35.148 257031 INFO cinder.volume.manager [req-b7fe089a-cf87-4c23-973a-5b021ed435e5 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Initialize volume connection completed successfully. >2018-08-29 05:35:39.780 257031 INFO cinder.volume.manager [req-425ab563-5cf6-43ec-aad4-739a26ea17e6 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Attaching volume 0a47c415-e3a0-4837-9a9c-d962cd795ced to instance 24be2681-f19b-4306-88d8-47bd9963617f at mountpoint /dev/vdb on host None. >2018-08-29 05:35:39.889 257031 INFO cinder.volume.manager [req-425ab563-5cf6-43ec-aad4-739a26ea17e6 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Attach volume completed successfully. >2018-08-29 05:35:40.494 257256 INFO cinder.volume.manager [req-5cd6f76d-ed1b-48fb-8d0a-70c0e5489517 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Deleted volume successfully. >2018-08-29 05:35:41.108 257031 INFO cinder.volume.manager [req-db4a2fd9-899d-43e7-84d8-43bde845591f 196daa0bca4b4e6e9f461e6d059bc482 fbed6f2bbe6f4c93b83ad6ddd320307a - default default] Deleted volume successfully. >2018-08-29 05:35:41.526 257031 INFO cinder.volume.manager [req-2075d0d0-eea8-4ec1-b752-cadb6b248c86 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Terminate volume connection completed successfully. >2018-08-29 05:35:41.687 257031 INFO cinder.volume.manager [req-eabade4b-e774-4732-a8c9-775c077e5b04 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Detaching volume 3e874d12-f73d-45b6-910b-75979b6ec9a8 from instance ada7ff8c-e009-4004-b0a1-c610663544d4. >2018-08-29 05:35:41.820 257031 INFO cinder.volume.manager [req-eabade4b-e774-4732-a8c9-775c077e5b04 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Detach volume completed successfully. >2018-08-29 05:35:44.447 257031 INFO cinder.volume.manager [req-5818b8de-4725-4e0b-8215-82699ba56356 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Create snapshot completed successfully >2018-08-29 05:35:47.835 257031 INFO cinder.volume.manager [req-4029f457-34b9-4b97-9ef7-e3fb5f9f4ae0 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:35:51.650 257031 INFO cinder.volume.manager [req-3d9c4f54-2acf-4dee-bff2-7ff7232c14f1 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Create snapshot completed successfully >2018-08-29 05:35:52.065 257031 INFO cinder.volume.manager [req-39971350-65cf-4219-b433-27f18bae7276 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Deleted volume successfully. >2018-08-29 05:35:53.405 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:35:53.516 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:35:55.522 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] initialize_connection: Did not find a preferred node for volume volume-29fa690f-9ad4-49ce-9856-5a8a8c727431. >2018-08-29 05:35:56.139 257031 INFO cinder.volume.manager [req-83302157-ff1d-4dca-8fdd-bbebd8334f21 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:35:58.137 257031 INFO cinder.volume.manager [req-91e6ad30-2eff-4086-a767-19c4b661a744 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Deleted volume successfully. >2018-08-29 05:36:00.549 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:36:02.167 257256 INFO cinder.volume.manager [req-9e0e050b-965a-4e55-92b9-e1e25a0e86ee 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] group_snapshot eae7e943-76e5-4527-897a-41fbbf7f5da9: deleting >2018-08-29 05:36:04.760 257031 INFO cinder.volume.manager [req-c3b50dd7-44ea-43b3-a948-d1022577a6bd 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Delete snapshot completed successfully. >2018-08-29 05:36:05.106 257031 INFO cinder.volume.manager [req-79ce0acd-e7c6-48c5-9c74-b843070d7156 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Delete snapshot completed successfully. >2018-08-29 05:36:05.498 257031 INFO cinder.volume.manager [req-3af0b33e-024a-4e82-ae23-a71ef2cacf75 668e74d2df7245f3a541e7d5f2b9f5ad d2fabfc7c9f5498c9c88525d8baf9b5f - default default] Deleted volume successfully. >2018-08-29 05:36:05.590 257031 INFO cinder.volume.manager [req-ada0cd79-2427-4763-bd7f-d3da30ce25c0 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Delete snapshot completed successfully. >2018-08-29 05:36:06.555 257031 INFO cinder.volume.manager [req-ae2b6779-8b5d-4f04-9752-98fb7599db2a 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:36:11.103 257031 INFO cinder.volume.manager [req-2586a14b-0b4c-48d1-aeec-3da55f2913fc 65f4af66bb0548129b83f5b2615d5a54 f24ea524a2594af9b9969f931712becf - default default] Deleted volume successfully. >2018-08-29 05:36:11.550 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:36:11.939 257256 WARNING cinder.quota [req-9e0e050b-965a-4e55-92b9-e1e25a0e86ee 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: snapshots_tempest-GroupsTest-volume-type-133687067 is set by the default quota flag: quota_snapshots_tempest-GroupsTest-volume-type-133687067, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:12.017 257256 INFO cinder.volume.manager [req-9e0e050b-965a-4e55-92b9-e1e25a0e86ee 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] group_snapshot eae7e943-76e5-4527-897a-41fbbf7f5da9: deleted successfully >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 29fa690f-9ad4-49ce-9856-5a8a8c727431: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:12.359 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:36:12.374 257256 WARNING cinder.volume.manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a78b76f0-d0e2-4829-a174-2a3baa72fd2a) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1145640167',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=29fa690f-9ad4-49ce-9856-5a8a8c727431,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2fb1149befe04d37b7cb9497bd0805fb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:57Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:57Z,user_id='c9c1cdb000824fcc8b648c7b267bc6d9',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5422e5d0>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1145640167',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=29fa690f-9ad4-49ce-9856-5a8a8c727431,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2fb1149befe04d37b7cb9497bd0805fb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:57Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:57Z,user_id='c9c1cdb000824fcc8b648c7b267bc6d9',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(29fa690f-9ad4-49ce-9856-5a8a8c727431),volume_id=29fa690f-9ad4-49ce-9856-5a8a8c727431,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab5422e5d0>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '29fa690f-9ad4-49ce-9856-5a8a8c727431', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54a847d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-29fa690f-9ad4-49ce-9856-5a8a8c727431', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1145640167',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=29fa690f-9ad4-49ce-9856-5a8a8c727431,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2fb1149befe04d37b7cb9497bd0805fb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:57Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:57Z,user_id='c9c1cdb000824fcc8b648c7b267bc6d9',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'c9c1cdb000824fcc8b648c7b267bc6d9', u'availability_zone': u'nova', u'reservations': [u'27376729-f3b0-4962-83fe-36a646a6cc2b', u'cead5cd6-8eea-47e2-8f57-e97fdda2c289'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'2fb1149befe04d37b7cb9497bd0805fb', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-tempest.common.compute-volume-1145640167'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'c9c1cdb000824fcc8b648c7b267bc6d9', u'availability_zone': u'nova', u'reservations': [u'27376729-f3b0-4962-83fe-36a646a6cc2b', u'cead5cd6-8eea-47e2-8f57-e97fdda2c289'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'2fb1149befe04d37b7cb9497bd0805fb', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-tempest.common.compute-volume-1145640167'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'29fa690f-9ad4-49ce-9856-5a8a8c727431', u'size': 1, u'display_name': u'tempest-tempest.common.compute-volume-1145640167', u'display_description': None, u'cluster_name': None, u'name_id': u'29fa690f-9ad4-49ce-9856-5a8a8c727431', u'volume_admin_metadata': [], u'project_id': u'2fb1149befe04d37b7cb9497bd0805fb', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-29fa690f-9ad4-49ce-9856-5a8a8c727431', u'user_id': u'c9c1cdb000824fcc8b648c7b267bc6d9', u'bootable': False, u'created_at': u'2018-08-29T05:33:56.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'29fa690f-9ad4-49ce-9856-5a8a8c727431', u'group_id': None}, u'user_id': u'c9c1cdb000824fcc8b648c7b267bc6d9', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab5422e5d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(29fa690f-9ad4-49ce-9856-5a8a8c727431),volume_id=29fa690f-9ad4-49ce-9856-5a8a8c727431,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1145640167',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=29fa690f-9ad4-49ce-9856-5a8a8c727431,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2fb1149befe04d37b7cb9497bd0805fb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:57Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:57Z,user_id='c9c1cdb000824fcc8b648c7b267bc6d9',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5422e5d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:33:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1145640167',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=29fa690f-9ad4-49ce-9856-5a8a8c727431,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2fb1149befe04d37b7cb9497bd0805fb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:33:57Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:33:57Z,user_id='c9c1cdb000824fcc8b648c7b267bc6d9',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:12.374 257256 ERROR cinder.volume.manager >2018-08-29 05:36:12.382 257256 WARNING cinder.volume.manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a78b76f0-d0e2-4829-a174-2a3baa72fd2a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:12.386 257256 WARNING cinder.volume.manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (005bc932-ab38-480c-b3c6-691d1e6cda97) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:12.390 257256 WARNING cinder.volume.manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f159dc0a-41a9-4ab2-92cf-f08ff6d3bcbb) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:12.412 257256 ERROR cinder.volume.flows.manager.create_volume [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Volume 29fa690f-9ad4-49ce-9856-5a8a8c727431: create failed >2018-08-29 05:36:12.414 257256 WARNING cinder.volume.manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d58e3de9-4a39-4385-a1e1-72d1d76abbe4) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:12.417 257256 WARNING cinder.volume.manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5a00796b-50d8-43eb-9fa6-c4338938d67d) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:12.420 257256 WARNING cinder.volume.manager [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Flow 'volume_create_manager' (d05266b0-6644-490a-847f-920312b64be7) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server [req-9811d0f8-bf29-4d24-b812-da1c46af87d0 c9c1cdb000824fcc8b648c7b267bc6d9 2fb1149befe04d37b7cb9497bd0805fb - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:12.434 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:36:13.694 257031 INFO cinder.volume.manager [req-89253765-34ea-46c0-a083-10d3404fe6c8 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:36:14.051 257256 INFO cinder.image.image_utils [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Image download 12.00 MB at 7.26 MB/s >2018-08-29 05:36:15.110 257256 WARNING cinder.quota [req-0477b653-eda9-44b1-8869-627b18f3a161 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-133687067 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-133687067, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:15.112 257256 WARNING cinder.quota [req-0477b653-eda9-44b1-8869-627b18f3a161 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-133687067 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-133687067, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:16.061 257256 INFO cinder.volume.manager [req-0477b653-eda9-44b1-8869-627b18f3a161 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group completed successfully. >2018-08-29 05:36:17.068 257256 INFO cinder.volume.manager [req-bbcfce3b-aef3-4ec5-ae71-0afa19dc5a27 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group1-1209055612: creating >2018-08-29 05:36:17.109 257256 INFO cinder.volume.manager [req-bbcfce3b-aef3-4ec5-ae71-0afa19dc5a27 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group1-1209055612: created successfully >2018-08-29 05:36:17.132 257256 INFO cinder.volume.manager [req-bbcfce3b-aef3-4ec5-ae71-0afa19dc5a27 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group completed successfully. >2018-08-29 05:36:18.376 257256 INFO cinder.volume.manager [req-3602e44b-e670-4fcf-8e66-f4d6903f3349 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group2-857438455: creating >2018-08-29 05:36:18.421 257256 INFO cinder.volume.manager [req-3602e44b-e670-4fcf-8e66-f4d6903f3349 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group2-857438455: created successfully >2018-08-29 05:36:18.449 257256 INFO cinder.volume.manager [req-3602e44b-e670-4fcf-8e66-f4d6903f3349 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group completed successfully. >2018-08-29 05:36:19.375 257031 INFO cinder.volume.manager [req-6e627125-17f2-4be2-8060-19bcd24a5a29 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:36:20.274 257256 INFO cinder.volume.flows.manager.create_volume [req-bcedeb33-da86-405a-a12f-27e1d4aee330 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume 9989722b-b45c-4bee-97bf-0276145c98ee: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9989722b-b45c-4bee-97bf-0276145c98ee'} >2018-08-29 05:36:22.501 257256 INFO cinder.volume.flows.manager.create_volume [req-bcedeb33-da86-405a-a12f-27e1d4aee330 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume volume-9989722b-b45c-4bee-97bf-0276145c98ee (9989722b-b45c-4bee-97bf-0276145c98ee): created successfully >2018-08-29 05:36:22.507 257256 INFO cinder.volume.manager [req-bcedeb33-da86-405a-a12f-27e1d4aee330 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Created volume successfully. >2018-08-29 05:36:25.740 257031 INFO cinder.volume.manager [req-9f411def-ba01-4b7e-a9b4-3a279b14e9a5 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:36:26.708 257256 WARNING cinder.quota [req-1ccd120a-6c0b-475e-83c1-444241967200 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-393127404 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-393127404, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:26.709 257256 WARNING cinder.quota [req-1ccd120a-6c0b-475e-83c1-444241967200 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-393127404 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-393127404, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:27.823 257256 INFO cinder.volume.manager [req-1ccd120a-6c0b-475e-83c1-444241967200 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group completed successfully. >2018-08-29 05:36:27.947 257256 INFO cinder.volume.manager [req-df63a8a5-75f1-4584-9dd9-c392d2a223bc 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group completed successfully. >2018-08-29 05:36:28.305 257256 INFO cinder.volume.manager [req-34ad7e19-6f61-437c-b0de-5df2ee52fa49 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-1085562958: creating >2018-08-29 05:36:28.339 257256 INFO cinder.volume.manager [req-34ad7e19-6f61-437c-b0de-5df2ee52fa49 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-1085562958: created successfully >2018-08-29 05:36:28.364 257256 INFO cinder.volume.manager [req-34ad7e19-6f61-437c-b0de-5df2ee52fa49 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group completed successfully. >2018-08-29 05:36:29.882 257256 INFO cinder.volume.flows.manager.create_volume [req-b6bd2443-3082-4038-b68b-121725539f72 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume de50a686-f73d-4666-b288-cf9d01be97d5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-de50a686-f73d-4666-b288-cf9d01be97d5'} >2018-08-29 05:36:31.309 257256 INFO cinder.volume.flows.manager.create_volume [req-b6bd2443-3082-4038-b68b-121725539f72 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume volume-de50a686-f73d-4666-b288-cf9d01be97d5 (de50a686-f73d-4666-b288-cf9d01be97d5): created successfully >2018-08-29 05:36:31.314 257256 INFO cinder.volume.manager [req-b6bd2443-3082-4038-b68b-121725539f72 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Created volume successfully. >2018-08-29 05:36:32.259 257256 INFO cinder.volume.manager [req-94b444b3-6c73-4bda-9ea9-a4eea9df00d9 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] GroupSnapshot 7be1414f-f46d-4a89-9f9a-430cda414d9a: creating. >2018-08-29 05:36:37.673 257031 INFO cinder.volume.manager [req-35d99fe8-5096-41c1-ae2f-0277144b70ab 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Terminate volume connection completed successfully. >2018-08-29 05:36:37.748 257256 INFO cinder.volume.manager [req-94b444b3-6c73-4bda-9ea9-a4eea9df00d9 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] group_snapshot 7be1414f-f46d-4a89-9f9a-430cda414d9a: created successfully >2018-08-29 05:36:38.029 257031 INFO cinder.volume.manager [req-1a6b17ad-070a-404a-8148-951392d8ff4a 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Detaching volume 0a47c415-e3a0-4837-9a9c-d962cd795ced from instance 24be2681-f19b-4306-88d8-47bd9963617f. >2018-08-29 05:36:38.124 257031 INFO cinder.volume.manager [req-1a6b17ad-070a-404a-8148-951392d8ff4a 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Detach volume completed successfully. >2018-08-29 05:36:38.398 257256 INFO cinder.volume.flows.manager.create_volume [req-85a40110-cb13-4c0c-b188-ef637aac8d60 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Volume 1da64ff5-9c85-4514-bfe9-288cb056a291: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1da64ff5-9c85-4514-bfe9-288cb056a291'} >2018-08-29 05:36:38.574 257256 INFO cinder.volume.manager [req-815e8abc-282c-45b7-be16-d00a88a2c90b 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] group_snapshot 7be1414f-f46d-4a89-9f9a-430cda414d9a: deleting >2018-08-29 05:36:39.848 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] initialize_connection: Did not find a preferred node for volume volume-56c77037-4f08-4104-8bfd-7bf9f7ede1e5. >2018-08-29 05:36:40.336 257256 INFO cinder.volume.flows.manager.create_volume [req-85a40110-cb13-4c0c-b188-ef637aac8d60 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Volume volume-1da64ff5-9c85-4514-bfe9-288cb056a291 (1da64ff5-9c85-4514-bfe9-288cb056a291): created successfully >2018-08-29 05:36:40.341 257256 INFO cinder.volume.manager [req-85a40110-cb13-4c0c-b188-ef637aac8d60 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Created volume successfully. >2018-08-29 05:36:42.751 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:36:46.063 257256 WARNING cinder.quota [req-815e8abc-282c-45b7-be16-d00a88a2c90b 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: snapshots_tempest-GroupsTest-volume-type-1625106350 is set by the default quota flag: quota_snapshots_tempest-GroupsTest-volume-type-1625106350, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:46.123 257256 INFO cinder.volume.manager [req-815e8abc-282c-45b7-be16-d00a88a2c90b 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] group_snapshot 7be1414f-f46d-4a89-9f9a-430cda414d9a: deleted successfully >2018-08-29 05:36:47.618 257031 INFO cinder.volume.manager [req-ff475991-774c-4178-9262-f18e9c7abf27 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:36:50.484 257256 WARNING cinder.quota [req-b01469cd-7e88-4650-b692-0b02dce0e6a4 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-1625106350 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-1625106350, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:50.486 257256 WARNING cinder.quota [req-b01469cd-7e88-4650-b692-0b02dce0e6a4 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-1625106350 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-1625106350, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:36:51.163 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:36:51.490 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-935d3c1f-4605-414c-847f-540c457a6a81 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] initialize_connection: Did not find a preferred node for volume volume-1da64ff5-9c85-4514-bfe9-288cb056a291. >2018-08-29 05:36:51.686 257256 INFO cinder.volume.manager [req-b01469cd-7e88-4650-b692-0b02dce0e6a4 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group completed successfully. >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 56c77037-4f08-4104-8bfd-7bf9f7ede1e5: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:52.352 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:36:52.365 257256 WARNING cinder.volume.manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (46ae2fbf-5bba-4da5-9faf-2048f8285799) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:35:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-311825773',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=56c77037-4f08-4104-8bfd-7bf9f7ede1e5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='109ae9965da740e696aebde8df123860',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:35:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:35:23Z,user_id='76c3645057814d4f897a714f4a43953f',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab540f13d0>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:35:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-311825773',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=56c77037-4f08-4104-8bfd-7bf9f7ede1e5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='109ae9965da740e696aebde8df123860',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:35:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:35:23Z,user_id='76c3645057814d4f897a714f4a43953f',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(56c77037-4f08-4104-8bfd-7bf9f7ede1e5),volume_id=56c77037-4f08-4104-8bfd-7bf9f7ede1e5,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab540f13d0>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '56c77037-4f08-4104-8bfd-7bf9f7ede1e5', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab540c9450>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-56c77037-4f08-4104-8bfd-7bf9f7ede1e5', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:35:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-311825773',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=56c77037-4f08-4104-8bfd-7bf9f7ede1e5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='109ae9965da740e696aebde8df123860',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:35:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:35:23Z,user_id='76c3645057814d4f897a714f4a43953f',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'76c3645057814d4f897a714f4a43953f', u'availability_zone': u'nova', u'reservations': [u'418f69c7-537b-4c5e-bf92-e89bfb89d618', u'de346d61-59b2-426b-b306-1ff1d20fc049'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'109ae9965da740e696aebde8df123860', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesImageMetadata-Volume-311825773'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'76c3645057814d4f897a714f4a43953f', u'availability_zone': u'nova', u'reservations': [u'418f69c7-537b-4c5e-bf92-e89bfb89d618', u'de346d61-59b2-426b-b306-1ff1d20fc049'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'109ae9965da740e696aebde8df123860', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesImageMetadata-Volume-311825773'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'56c77037-4f08-4104-8bfd-7bf9f7ede1e5', u'size': 1, u'display_name': u'tempest-VolumesImageMetadata-Volume-311825773', u'display_description': None, u'cluster_name': None, u'name_id': u'56c77037-4f08-4104-8bfd-7bf9f7ede1e5', u'volume_admin_metadata': [], u'project_id': u'109ae9965da740e696aebde8df123860', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-56c77037-4f08-4104-8bfd-7bf9f7ede1e5', u'user_id': u'76c3645057814d4f897a714f4a43953f', u'bootable': False, u'created_at': u'2018-08-29T05:35:22.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'56c77037-4f08-4104-8bfd-7bf9f7ede1e5', u'group_id': None}, u'user_id': u'76c3645057814d4f897a714f4a43953f', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab540f13d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(56c77037-4f08-4104-8bfd-7bf9f7ede1e5),volume_id=56c77037-4f08-4104-8bfd-7bf9f7ede1e5,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:35:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-311825773',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=56c77037-4f08-4104-8bfd-7bf9f7ede1e5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='109ae9965da740e696aebde8df123860',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:35:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:35:23Z,user_id='76c3645057814d4f897a714f4a43953f',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab540f13d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:35:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-311825773',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=56c77037-4f08-4104-8bfd-7bf9f7ede1e5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='109ae9965da740e696aebde8df123860',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:35:23Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:35:23Z,user_id='76c3645057814d4f897a714f4a43953f',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:52.365 257256 ERROR cinder.volume.manager >2018-08-29 05:36:52.372 257256 WARNING cinder.volume.manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (46ae2fbf-5bba-4da5-9faf-2048f8285799) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:52.375 257256 WARNING cinder.volume.manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fb404b18-63e0-4ba4-8319-43e182f64623) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:52.378 257256 WARNING cinder.volume.manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f46a7c3f-96fd-49cf-a651-718092ae8c7d) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:52.385 257256 INFO cinder.volume.manager [req-935d3c1f-4605-414c-847f-540c457a6a81 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Initialize volume connection completed successfully. >2018-08-29 05:36:52.389 257256 ERROR cinder.volume.flows.manager.create_volume [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Volume 56c77037-4f08-4104-8bfd-7bf9f7ede1e5: create failed >2018-08-29 05:36:52.390 257256 WARNING cinder.volume.manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8b67b6d7-74ce-4ba8-9656-2a6edddff9ff) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:52.393 257256 WARNING cinder.volume.manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (88d94132-8be7-4178-93bc-6100ef2e39c4) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:36:52.395 257256 WARNING cinder.volume.manager [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Flow 'volume_create_manager' (2e47aced-57b8-4a26-98a2-8f5dbc9e5bfd) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server [req-f2515296-8d9a-4261-8a22-6a6c24d8cf4f 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:36:52.400 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:36:52.459 257256 INFO cinder.volume.manager [req-b255c043-5346-4af6-a0ff-716a8d443cc0 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-2066836758: creating >2018-08-29 05:36:52.488 257256 INFO cinder.volume.manager [req-b255c043-5346-4af6-a0ff-716a8d443cc0 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Group tempest-Group-2066836758: created successfully >2018-08-29 05:36:52.511 257256 INFO cinder.volume.manager [req-b255c043-5346-4af6-a0ff-716a8d443cc0 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Create group completed successfully. >2018-08-29 05:36:53.536 257256 INFO cinder.volume.flows.manager.create_volume [req-9831983f-2a02-47fd-8026-e29210c029f7 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Volume 5da5946c-1fb5-4789-a264-8cceeefb4b57: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5da5946c-1fb5-4789-a264-8cceeefb4b57'} >2018-08-29 05:36:54.005 257031 INFO cinder.volume.manager [req-b6219f6e-4e05-4b9d-9496-554026547f68 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:36:54.055 257256 INFO cinder.volume.flows.manager.create_volume [req-617e936b-a366-4c70-b06d-815899b5b400 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume d8e5b774-b035-422b-a67c-5cfd85358d3c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d8e5b774-b035-422b-a67c-5cfd85358d3c'} >2018-08-29 05:36:55.400 257256 INFO cinder.volume.flows.manager.create_volume [req-9831983f-2a02-47fd-8026-e29210c029f7 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Volume volume-5da5946c-1fb5-4789-a264-8cceeefb4b57 (5da5946c-1fb5-4789-a264-8cceeefb4b57): created successfully >2018-08-29 05:36:55.405 257256 INFO cinder.volume.manager [req-9831983f-2a02-47fd-8026-e29210c029f7 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Created volume successfully. >2018-08-29 05:36:56.121 257256 INFO cinder.volume.flows.manager.create_volume [req-617e936b-a366-4c70-b06d-815899b5b400 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume volume-d8e5b774-b035-422b-a67c-5cfd85358d3c (d8e5b774-b035-422b-a67c-5cfd85358d3c): created successfully >2018-08-29 05:36:56.125 257256 INFO cinder.volume.manager [req-617e936b-a366-4c70-b06d-815899b5b400 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Created volume successfully. >2018-08-29 05:36:56.896 257256 INFO cinder.volume.flows.manager.create_volume [req-e4e66f25-49ac-4647-9219-fb735cacb977 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume 0088d497-4351-4b08-9c38-5ef141c73789: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0088d497-4351-4b08-9c38-5ef141c73789'} >2018-08-29 05:36:57.727 257256 INFO cinder.volume.manager [req-7a749f11-6441-4669-88c7-6fb98b1e51c1 76c3645057814d4f897a714f4a43953f 109ae9965da740e696aebde8df123860 - default default] Deleted volume successfully. >2018-08-29 05:36:58.237 257256 INFO cinder.volume.flows.manager.create_volume [req-e4e66f25-49ac-4647-9219-fb735cacb977 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Volume volume-0088d497-4351-4b08-9c38-5ef141c73789 (0088d497-4351-4b08-9c38-5ef141c73789): created successfully >2018-08-29 05:36:58.241 257256 INFO cinder.volume.manager [req-e4e66f25-49ac-4647-9219-fb735cacb977 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Created volume successfully. >2018-08-29 05:36:59.190 257256 INFO cinder.volume.manager [req-45a003d6-9ea7-4105-92e8-f60d36f23fd8 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Update group completed successfully. >2018-08-29 05:36:59.228 257256 INFO cinder.volume.manager [req-92da0fb9-8251-4976-99d9-ff8d263bbf54 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Attaching volume 1da64ff5-9c85-4514-bfe9-288cb056a291 to instance 25eac5f8-8491-4586-b33a-148adce49325 at mountpoint /dev/vdb on host None. >2018-08-29 05:36:59.316 257256 INFO cinder.volume.manager [req-92da0fb9-8251-4976-99d9-ff8d263bbf54 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Attach volume completed successfully. >2018-08-29 05:36:59.605 257031 INFO cinder.volume.manager [req-c6911650-8721-4335-ab1b-752392b26380 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:37:00.633 257256 INFO cinder.volume.manager [req-26d84608-a2ec-4c24-badf-50042d99d8c1 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Update group completed successfully. >2018-08-29 05:37:05.927 257256 WARNING cinder.quota [req-3fb24142-9881-48a8-b7a5-70153b1ec6a2 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-1489249786 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-1489249786, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:37:05.929 257256 WARNING cinder.quota [req-3fb24142-9881-48a8-b7a5-70153b1ec6a2 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-1489249786 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-1489249786, it is now deprecated. Please use the default quota class for default quota.: NotImplementedError >2018-08-29 05:37:06.590 257256 INFO cinder.volume.manager [req-3fb24142-9881-48a8-b7a5-70153b1ec6a2 1f920de474ee4f46af203fbcf918d787 5d2022cefc3b469c86b78ca369bfd43b - default default] Delete group completed successfully. >2018-08-29 05:37:14.646 257256 INFO cinder.volume.manager [req-16f127ae-4b46-4414-a1ad-53be90586193 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Terminate volume connection completed successfully. >2018-08-29 05:37:14.898 257256 INFO cinder.volume.manager [req-21aa0aed-4b73-485f-8705-6af271064a25 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Detaching volume 1da64ff5-9c85-4514-bfe9-288cb056a291 from instance 25eac5f8-8491-4586-b33a-148adce49325. >2018-08-29 05:37:14.997 257256 INFO cinder.volume.manager [req-21aa0aed-4b73-485f-8705-6af271064a25 d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Detach volume completed successfully. >2018-08-29 05:37:16.886 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-06b7d8aa-f93f-4dbd-bc7e-7254398f3243 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] initialize_connection: Did not find a preferred node for volume volume-5da5946c-1fb5-4789-a264-8cceeefb4b57. >2018-08-29 05:37:17.288 257256 INFO cinder.volume.manager [req-06b7d8aa-f93f-4dbd-bc7e-7254398f3243 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Initialize volume connection completed successfully. >2018-08-29 05:37:17.984 257256 INFO cinder.volume.flows.manager.create_volume [req-4361c137-842e-4e3a-8bdc-5a578500f724 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] Volume e6a3ad30-2b62-4aec-a91c-16f99f14a618: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e6a3ad30-2b62-4aec-a91c-16f99f14a618'} >2018-08-29 05:37:19.510 257256 INFO cinder.volume.flows.manager.create_volume [req-4361c137-842e-4e3a-8bdc-5a578500f724 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] Volume volume-e6a3ad30-2b62-4aec-a91c-16f99f14a618 (e6a3ad30-2b62-4aec-a91c-16f99f14a618): created successfully >2018-08-29 05:37:19.516 257256 INFO cinder.volume.manager [req-4361c137-842e-4e3a-8bdc-5a578500f724 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] Created volume successfully. >2018-08-29 05:37:20.094 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b733870c-b640-447d-9e38-4ade6eb8a121 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] initialize_connection: Did not find a preferred node for volume volume-0a47c415-e3a0-4837-9a9c-d962cd795ced. >2018-08-29 05:37:20.564 257031 INFO cinder.volume.manager [req-b733870c-b640-447d-9e38-4ade6eb8a121 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Initialize volume connection completed successfully. >2018-08-29 05:37:22.117 257256 INFO cinder.volume.manager [req-55373f1f-d0f1-4933-82ce-76b3377cf389 6d6b3aa6a40e4d7a831d322cb7e9fc0b a84f0234d18b49a6b16eac87e8ef490e - default default] Transfer volume completed successfully. >2018-08-29 05:37:23.374 257256 INFO cinder.volume.manager [req-a8dc365f-feb3-4115-8bd3-20e2745e765a 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Attaching volume 5da5946c-1fb5-4789-a264-8cceeefb4b57 to instance 77900bcb-ff40-452f-9f25-0152cd464485 at mountpoint /dev/vdb on host None. >2018-08-29 05:37:23.449 257256 INFO cinder.volume.manager [req-a8dc365f-feb3-4115-8bd3-20e2745e765a 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Attach volume completed successfully. >2018-08-29 05:37:25.838 257256 INFO cinder.volume.manager [req-a3ede130-fdef-44d8-a56b-e6e135c1fe1b 044974c5e61d4323b382ecbc95bb2c40 bdd1d3f507004a2083fc649781e6e1b5 - default default] Deleted volume successfully. >2018-08-29 05:37:26.443 257256 INFO cinder.volume.flows.manager.create_volume [req-332c45fc-cae5-40ad-8da7-45716344446c 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] Volume e170a918-fb59-41a0-bb84-42ccf3b78ffd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e170a918-fb59-41a0-bb84-42ccf3b78ffd'} >2018-08-29 05:37:27.339 257031 INFO cinder.volume.manager [req-1c7a2dc0-dac7-4785-ab2c-122682428170 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Attaching volume 0a47c415-e3a0-4837-9a9c-d962cd795ced to instance 3fc939f4-215a-4552-bf54-042219ad3376 at mountpoint /dev/vdb on host None. >2018-08-29 05:37:27.418 257031 INFO cinder.volume.manager [req-1c7a2dc0-dac7-4785-ab2c-122682428170 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Attach volume completed successfully. >2018-08-29 05:37:27.785 257256 INFO cinder.volume.flows.manager.create_volume [req-332c45fc-cae5-40ad-8da7-45716344446c 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] Volume volume-e170a918-fb59-41a0-bb84-42ccf3b78ffd (e170a918-fb59-41a0-bb84-42ccf3b78ffd): created successfully >2018-08-29 05:37:27.789 257256 INFO cinder.volume.manager [req-332c45fc-cae5-40ad-8da7-45716344446c 45640fc1d3824a37b597ccee4d130e61 87ddfcd74cb240e386f6f04a4d7cbfa5 - default default] Created volume successfully. >2018-08-29 05:37:32.562 257256 INFO cinder.volume.manager [req-0687de64-3426-4e16-acb3-bbc5de93165e 044974c5e61d4323b382ecbc95bb2c40 bdd1d3f507004a2083fc649781e6e1b5 - default default] Deleted volume successfully. >2018-08-29 05:37:38.403 257031 INFO cinder.volume.manager [req-66bab5c4-6cf1-470c-8183-6640719a3205 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:37:49.975 257031 INFO cinder.volume.manager [req-88d4d2b4-0c6c-4962-a12e-958959274760 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:37:52.222 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-805ce108-9120-4574-b8e6-2cdca79ccf28 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] initialize_connection: Did not find a preferred node for volume volume-5f09ab9a-88b2-4fff-a05c-c92fe23ba086. >2018-08-29 05:37:54.351 257256 INFO cinder.volume.manager [req-805ce108-9120-4574-b8e6-2cdca79ccf28 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Initialize volume connection completed successfully. >2018-08-29 05:37:57.383 257031 INFO cinder.volume.manager [req-d1e14a8d-4d8c-4570-8297-86ca7aeb2ec3 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:38:01.443 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-805ce108-9120-4574-b8e6-2cdca79ccf28 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:38:02.327 257256 INFO cinder.volume.manager [req-805ce108-9120-4574-b8e6-2cdca79ccf28 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Terminate volume connection completed successfully. >2018-08-29 05:38:04.174 257031 INFO cinder.volume.manager [req-94b0adca-cbc1-4edc-b978-354a18a5130f 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:38:06.350 257256 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 0.10 sec >2018-08-29 05:38:08.851 257256 INFO cinder.volume.manager [req-805ce108-9120-4574-b8e6-2cdca79ccf28 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Deleted volume successfully. >2018-08-29 05:38:13.630 257031 INFO cinder.volume.manager [req-c26a802a-77b0-433f-8bbc-a50c76f75e7a 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:38:15.293 257256 INFO cinder.volume.manager [req-16ebbaf8-9be4-4c13-8588-e98899fe5ffd 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Terminate volume connection completed successfully. >2018-08-29 05:38:15.605 257256 INFO cinder.volume.manager [req-59ffc10f-916d-44c5-a9b8-142494dd89e6 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Detaching volume 5da5946c-1fb5-4789-a264-8cceeefb4b57 from instance 77900bcb-ff40-452f-9f25-0152cd464485. >2018-08-29 05:38:15.684 257256 INFO cinder.volume.manager [req-59ffc10f-916d-44c5-a9b8-142494dd89e6 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Detach volume completed successfully. >2018-08-29 05:38:20.290 257256 INFO cinder.volume.flows.manager.create_volume [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Volume 50478d25-9e61-4cfb-b6b3-3b9c8283f66c: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-50478d25-9e61-4cfb-b6b3-3b9c8283f66c', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab545e6150>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:38:21.095 257256 INFO cinder.volume.manager [req-fd3df871-da93-4b50-a414-f43edc7b849a 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Deleted volume successfully. >2018-08-29 05:38:21.133 257256 INFO cinder.image.image_utils [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-29 05:38:25.380 257031 INFO cinder.volume.manager [req-ba64414e-5fe9-4994-bd82-25aee6f8bd85 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Terminate volume connection completed successfully. >2018-08-29 05:38:25.532 257031 INFO cinder.volume.manager [req-9e4a54e5-1bdc-4ef2-b253-1300f90b4826 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Detaching volume 0a47c415-e3a0-4837-9a9c-d962cd795ced from instance 3fc939f4-215a-4552-bf54-042219ad3376. >2018-08-29 05:38:25.613 257031 INFO cinder.volume.manager [req-9e4a54e5-1bdc-4ef2-b253-1300f90b4826 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Detach volume completed successfully. >2018-08-29 05:38:26.402 257256 INFO cinder.volume.manager [req-b22f18a8-7874-4f27-9cc5-7100fcce4d5c d426417b489f479fbed4948faecc5589 cc816dd5f8784019929aa23f14b9e5c6 - default default] Deleted volume successfully. >2018-08-29 05:38:28.735 257256 INFO cinder.volume.flows.manager.create_volume [req-97749699-14c8-45d5-b8c0-b222b09f9e4c 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Volume a4cd2129-d0b0-43e4-b0ca-121bf3d8cc02: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a4cd2129-d0b0-43e4-b0ca-121bf3d8cc02'} >2018-08-29 05:38:30.090 257256 INFO cinder.volume.flows.manager.create_volume [req-97749699-14c8-45d5-b8c0-b222b09f9e4c 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Volume volume-a4cd2129-d0b0-43e4-b0ca-121bf3d8cc02 (a4cd2129-d0b0-43e4-b0ca-121bf3d8cc02): created successfully >2018-08-29 05:38:30.094 257256 INFO cinder.volume.manager [req-97749699-14c8-45d5-b8c0-b222b09f9e4c 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Created volume successfully. >2018-08-29 05:38:36.927 257031 INFO cinder.volume.manager [req-780c3e31-f300-4dfd-a4a0-76c8529584db 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Terminate volume connection completed successfully. >2018-08-29 05:38:37.572 257031 INFO cinder.volume.manager [req-ccca7e11-b221-4e0e-8dd8-790ee68b52eb 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Detaching volume 82c26162-8606-4b7b-91a7-6b60e79ee19e from instance 3b0b563f-7e06-47eb-9d6e-ab0d84389a4a. >2018-08-29 05:38:37.659 257031 INFO cinder.volume.manager [req-ccca7e11-b221-4e0e-8dd8-790ee68b52eb 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Detach volume completed successfully. >2018-08-29 05:38:41.803 257031 INFO cinder.volume.manager [req-14fbdeb3-39f6-4ff1-99e6-9e6ca64628de 480b7532226843f9baca86763712a91c 65f92849119e445a919ff81261d71827 - default default] Deleted volume successfully. >2018-08-29 05:38:42.185 257256 INFO cinder.volume.manager [req-dff4f1cf-c84f-49e2-99c3-033e2751050c 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Create snapshot completed successfully >2018-08-29 05:38:42.801 257256 INFO cinder.volume.flows.manager.create_volume [req-c9f80249-1a18-42ae-a1bf-545afc98b6ac 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Volume 4d4067c3-55ae-4d83-9c46-628d02423918: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': '6e0fc9ac-f901-4985-ab4f-b5a289192344', 'volume_name': 'volume-4d4067c3-55ae-4d83-9c46-628d02423918'} >2018-08-29 05:38:44.139 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] initialize_connection: Did not find a preferred node for volume volume-50478d25-9e61-4cfb-b6b3-3b9c8283f66c. >2018-08-29 05:38:46.905 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:38:48.462 257256 INFO cinder.volume.flows.manager.create_volume [req-6727729f-2de7-4ce8-872b-b4542d50d612 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume 3ed11abe-a87e-4ff9-8cf0-0e4e39e605ae: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3ed11abe-a87e-4ff9-8cf0-0e4e39e605ae'} >2018-08-29 05:38:49.856 257256 INFO cinder.volume.flows.manager.create_volume [req-6727729f-2de7-4ce8-872b-b4542d50d612 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume volume-3ed11abe-a87e-4ff9-8cf0-0e4e39e605ae (3ed11abe-a87e-4ff9-8cf0-0e4e39e605ae): created successfully >2018-08-29 05:38:49.860 257256 INFO cinder.volume.manager [req-6727729f-2de7-4ce8-872b-b4542d50d612 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Created volume successfully. >2018-08-29 05:38:50.760 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 50478d25-9e61-4cfb-b6b3-3b9c8283f66c: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:38:51.196 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:38:51.205 257256 WARNING cinder.volume.manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7608b5db-e278-4fd1-be26-8b02ee328c3f) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:38:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-262560072',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=50478d25-9e61-4cfb-b6b3-3b9c8283f66c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a2829c6e575b491a8a06d6ac41e0dc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:38:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:38:19Z,user_id='370e7a8931ce4548baf458ddf8695e8b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5402a150>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:38:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-262560072',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=50478d25-9e61-4cfb-b6b3-3b9c8283f66c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a2829c6e575b491a8a06d6ac41e0dc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:38:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:38:19Z,user_id='370e7a8931ce4548baf458ddf8695e8b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(50478d25-9e61-4cfb-b6b3-3b9c8283f66c),volume_id=50478d25-9e61-4cfb-b6b3-3b9c8283f66c,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab5402a150>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '50478d25-9e61-4cfb-b6b3-3b9c8283f66c', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab545e6150>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-50478d25-9e61-4cfb-b6b3-3b9c8283f66c', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:38:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-262560072',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=50478d25-9e61-4cfb-b6b3-3b9c8283f66c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a2829c6e575b491a8a06d6ac41e0dc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:38:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:38:19Z,user_id='370e7a8931ce4548baf458ddf8695e8b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'370e7a8931ce4548baf458ddf8695e8b', u'availability_zone': u'nova', u'reservations': [u'b0b8fe8b-ace4-42bb-a96b-eb7b42532a89', u'776a8a3a-103c-4fe7-baee-63e4139c01ff'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a2829c6e575b491a8a06d6ac41e0dc8b', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-262560072'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'370e7a8931ce4548baf458ddf8695e8b', u'availability_zone': u'nova', u'reservations': [u'b0b8fe8b-ace4-42bb-a96b-eb7b42532a89', u'776a8a3a-103c-4fe7-baee-63e4139c01ff'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'a2829c6e575b491a8a06d6ac41e0dc8b', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-262560072'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'50478d25-9e61-4cfb-b6b3-3b9c8283f66c', u'size': 1, u'display_name': u'tempest-VolumesBackupsTest-Volume-262560072', u'display_description': None, u'cluster_name': None, u'name_id': u'50478d25-9e61-4cfb-b6b3-3b9c8283f66c', u'volume_admin_metadata': [], u'project_id': u'a2829c6e575b491a8a06d6ac41e0dc8b', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-50478d25-9e61-4cfb-b6b3-3b9c8283f66c', u'user_id': u'370e7a8931ce4548baf458ddf8695e8b', u'bootable': False, u'created_at': u'2018-08-29T05:38:19.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'50478d25-9e61-4cfb-b6b3-3b9c8283f66c', u'group_id': None}, u'user_id': u'370e7a8931ce4548baf458ddf8695e8b', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab5402a150>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(50478d25-9e61-4cfb-b6b3-3b9c8283f66c),volume_id=50478d25-9e61-4cfb-b6b3-3b9c8283f66c,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:38:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-262560072',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=50478d25-9e61-4cfb-b6b3-3b9c8283f66c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a2829c6e575b491a8a06d6ac41e0dc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:38:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:38:19Z,user_id='370e7a8931ce4548baf458ddf8695e8b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5402a150>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:38:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-262560072',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=50478d25-9e61-4cfb-b6b3-3b9c8283f66c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a2829c6e575b491a8a06d6ac41e0dc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:38:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:38:19Z,user_id='370e7a8931ce4548baf458ddf8695e8b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:38:51.205 257256 ERROR cinder.volume.manager >2018-08-29 05:38:51.211 257256 WARNING cinder.volume.manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7608b5db-e278-4fd1-be26-8b02ee328c3f) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:38:51.214 257256 WARNING cinder.volume.manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c136d534-7d89-40b3-b991-6a5860db2fa6) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:38:51.217 257256 WARNING cinder.volume.manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8bef9308-9c20-455e-ac3b-7414e4e3e0ff) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:38:51.223 257256 ERROR cinder.volume.flows.manager.create_volume [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Volume 50478d25-9e61-4cfb-b6b3-3b9c8283f66c: create failed >2018-08-29 05:38:51.224 257256 WARNING cinder.volume.manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (75f9f56f-31f7-4e7f-a6c2-6a76f1b504f6) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:38:51.227 257256 WARNING cinder.volume.manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c07cf58f-570e-44f4-8803-c864bbfb9cab) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:38:51.229 257256 WARNING cinder.volume.manager [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Flow 'volume_create_manager' (2c4893a3-f15d-48dd-a4ff-26bf17a19e81) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server [req-be69af73-087e-4e11-8f99-a729071dff49 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:38:51.234 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:38:51.933 257031 INFO cinder.volume.flows.manager.create_volume [req-df3d7f52-3595-42b8-bb2b-e12d67b7bf70 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Volume b17a945e-9d23-4582-b653-0d0ef3cad852: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b17a945e-9d23-4582-b653-0d0ef3cad852'} >2018-08-29 05:38:52.841 257256 INFO cinder.volume.flows.manager.create_volume [req-eec79e16-5134-4a83-a0e0-b72e91bbeaad 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume 228ec503-0041-423d-a641-6a69fbd7bd15: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-228ec503-0041-423d-a641-6a69fbd7bd15'} >2018-08-29 05:38:53.494 257031 INFO cinder.volume.flows.manager.create_volume [req-df3d7f52-3595-42b8-bb2b-e12d67b7bf70 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Volume volume-b17a945e-9d23-4582-b653-0d0ef3cad852 (b17a945e-9d23-4582-b653-0d0ef3cad852): created successfully >2018-08-29 05:38:53.498 257031 INFO cinder.volume.manager [req-df3d7f52-3595-42b8-bb2b-e12d67b7bf70 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Created volume successfully. >2018-08-29 05:38:54.572 257256 INFO cinder.volume.flows.manager.create_volume [req-eec79e16-5134-4a83-a0e0-b72e91bbeaad 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume volume-228ec503-0041-423d-a641-6a69fbd7bd15 (228ec503-0041-423d-a641-6a69fbd7bd15): created successfully >2018-08-29 05:38:54.577 257256 INFO cinder.volume.manager [req-eec79e16-5134-4a83-a0e0-b72e91bbeaad 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Created volume successfully. >2018-08-29 05:39:00.850 257256 INFO cinder.volume.manager [req-a4ea089f-a9a4-427a-8fc1-7021de69fe31 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Deleted volume successfully. >2018-08-29 05:39:03.876 257031 INFO cinder.volume.flows.manager.create_volume [req-1c72bfb2-9180-4fe8-a45a-35e57a7c600a 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Volume bb1daae0-ea69-413c-b52d-a93e806a27a2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bb1daae0-ea69-413c-b52d-a93e806a27a2'} >2018-08-29 05:39:04.417 257256 INFO cinder.volume.manager [req-7f23a7ed-343d-443f-ad36-9ae7b4f438f5 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Deleted volume successfully. >2018-08-29 05:39:04.419 257031 INFO cinder.volume.flows.manager.create_volume [req-6d57ad16-1875-4fdf-a9c2-c41a40677256 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume 0f940c75-b3ba-4e1b-b3ff-4876fdf86ea8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0f940c75-b3ba-4e1b-b3ff-4876fdf86ea8'} >2018-08-29 05:39:05.688 257031 INFO cinder.volume.flows.manager.create_volume [req-1c72bfb2-9180-4fe8-a45a-35e57a7c600a 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Volume volume-bb1daae0-ea69-413c-b52d-a93e806a27a2 (bb1daae0-ea69-413c-b52d-a93e806a27a2): created successfully >2018-08-29 05:39:05.692 257031 INFO cinder.volume.manager [req-1c72bfb2-9180-4fe8-a45a-35e57a7c600a 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Created volume successfully. >2018-08-29 05:39:06.454 257031 INFO cinder.volume.flows.manager.create_volume [req-6d57ad16-1875-4fdf-a9c2-c41a40677256 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume volume-0f940c75-b3ba-4e1b-b3ff-4876fdf86ea8 (0f940c75-b3ba-4e1b-b3ff-4876fdf86ea8): created successfully >2018-08-29 05:39:06.459 257031 INFO cinder.volume.manager [req-6d57ad16-1875-4fdf-a9c2-c41a40677256 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Created volume successfully. >2018-08-29 05:39:07.185 257031 INFO cinder.volume.flows.manager.create_volume [req-e57910a0-c77b-4c20-8e52-a47b395c7f8f 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume ce11d692-e5cd-42f6-bd6b-6ed9adc59d1b: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ce11d692-e5cd-42f6-bd6b-6ed9adc59d1b', 'source_volid': '0f940c75-b3ba-4e1b-b3ff-4876fdf86ea8', 'source_volstatus': u'available'} >2018-08-29 05:39:12.539 257256 INFO cinder.volume.flows.manager.create_volume [req-1595d3c4-7bdc-4a07-a34e-27ce02875c4a fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Volume ffbfed73-40f6-4ecc-87cb-612436470a82: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ffbfed73-40f6-4ecc-87cb-612436470a82'} >2018-08-29 05:39:14.634 257256 INFO cinder.volume.flows.manager.create_volume [req-1595d3c4-7bdc-4a07-a34e-27ce02875c4a fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Volume volume-ffbfed73-40f6-4ecc-87cb-612436470a82 (ffbfed73-40f6-4ecc-87cb-612436470a82): created successfully >2018-08-29 05:39:14.639 257256 INFO cinder.volume.manager [req-1595d3c4-7bdc-4a07-a34e-27ce02875c4a fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Created volume successfully. >2018-08-29 05:39:16.073 257256 INFO cinder.volume.flows.manager.create_volume [req-d2da2811-54c1-4979-9f32-7e1984fb2029 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Volume f27d909b-d34b-4896-9f41-c5ba4073a046: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f27d909b-d34b-4896-9f41-c5ba4073a046'} >2018-08-29 05:39:16.460 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-6834e566-a67d-4de8-80b6-a52358558ae2 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] initialize_connection: Did not find a preferred node for volume volume-bb1daae0-ea69-413c-b52d-a93e806a27a2. >2018-08-29 05:39:17.521 257031 INFO cinder.volume.manager [req-6834e566-a67d-4de8-80b6-a52358558ae2 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Initialize volume connection completed successfully. >2018-08-29 05:39:18.572 257031 INFO cinder.volume.flows.manager.create_volume [req-e57910a0-c77b-4c20-8e52-a47b395c7f8f 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume volume-ce11d692-e5cd-42f6-bd6b-6ed9adc59d1b (ce11d692-e5cd-42f6-bd6b-6ed9adc59d1b): created successfully >2018-08-29 05:39:18.576 257031 INFO cinder.volume.manager [req-e57910a0-c77b-4c20-8e52-a47b395c7f8f 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Created volume successfully. >2018-08-29 05:39:18.704 257256 INFO cinder.volume.flows.manager.create_volume [req-d2da2811-54c1-4979-9f32-7e1984fb2029 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Volume volume-f27d909b-d34b-4896-9f41-c5ba4073a046 (f27d909b-d34b-4896-9f41-c5ba4073a046): created successfully >2018-08-29 05:39:18.708 257256 INFO cinder.volume.manager [req-d2da2811-54c1-4979-9f32-7e1984fb2029 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Created volume successfully. >2018-08-29 05:39:19.826 257031 INFO cinder.volume.flows.manager.create_volume [req-43d107c3-0060-4b63-b97a-ba50eb0f069a 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume 8fe5169b-43b9-47e8-a3a5-e27360fd40ed: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8fe5169b-43b9-47e8-a3a5-e27360fd40ed'} >2018-08-29 05:39:20.417 257256 INFO cinder.volume.flows.manager.create_volume [req-b91a5bd3-acbe-4e33-9bff-f29d712c38b4 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Volume d0f8d23c-68c8-4ee9-95e6-ac39ef6b243e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d0f8d23c-68c8-4ee9-95e6-ac39ef6b243e'} >2018-08-29 05:39:21.876 257031 INFO cinder.volume.flows.manager.create_volume [req-43d107c3-0060-4b63-b97a-ba50eb0f069a 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume volume-8fe5169b-43b9-47e8-a3a5-e27360fd40ed (8fe5169b-43b9-47e8-a3a5-e27360fd40ed): created successfully >2018-08-29 05:39:21.880 257031 INFO cinder.volume.manager [req-43d107c3-0060-4b63-b97a-ba50eb0f069a 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Created volume successfully. >2018-08-29 05:39:22.476 257031 INFO cinder.volume.manager [req-262b4cd8-be02-4efe-bc79-9d7fda841d20 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Attaching volume bb1daae0-ea69-413c-b52d-a93e806a27a2 to instance fd4efa61-941b-4112-88ba-f68eb7aa9804 at mountpoint /dev/vdb on host None. >2018-08-29 05:39:22.552 257031 INFO cinder.volume.manager [req-262b4cd8-be02-4efe-bc79-9d7fda841d20 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Attach volume completed successfully. >2018-08-29 05:39:22.927 257256 INFO cinder.volume.flows.manager.create_volume [req-b91a5bd3-acbe-4e33-9bff-f29d712c38b4 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Volume volume-d0f8d23c-68c8-4ee9-95e6-ac39ef6b243e (d0f8d23c-68c8-4ee9-95e6-ac39ef6b243e): created successfully >2018-08-29 05:39:22.931 257256 INFO cinder.volume.manager [req-b91a5bd3-acbe-4e33-9bff-f29d712c38b4 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Created volume successfully. >2018-08-29 05:39:22.934 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-98535e19-6422-466a-8566-a741b4ef8bcb 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] initialize_connection: Did not find a preferred node for volume volume-b17a945e-9d23-4582-b653-0d0ef3cad852. >2018-08-29 05:39:24.601 257031 INFO cinder.volume.flows.manager.create_volume [req-09e2c50a-5230-485f-ad31-e3662eb579d7 f7d98a03c05f4120bc04f97eaecc86ff 9fdee1ed6a78476089ea061ece7ec69e - default default] Volume 54d22db6-73b9-4296-8bcd-d5169bdaa688: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-54d22db6-73b9-4296-8bcd-d5169bdaa688'} >2018-08-29 05:39:25.506 257031 INFO cinder.volume.manager [req-98535e19-6422-466a-8566-a741b4ef8bcb 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Initialize volume connection completed successfully. >2018-08-29 05:39:26.690 257031 INFO cinder.volume.flows.manager.create_volume [req-09e2c50a-5230-485f-ad31-e3662eb579d7 f7d98a03c05f4120bc04f97eaecc86ff 9fdee1ed6a78476089ea061ece7ec69e - default default] Volume volume-54d22db6-73b9-4296-8bcd-d5169bdaa688 (54d22db6-73b9-4296-8bcd-d5169bdaa688): created successfully >2018-08-29 05:39:26.694 257031 INFO cinder.volume.manager [req-09e2c50a-5230-485f-ad31-e3662eb579d7 f7d98a03c05f4120bc04f97eaecc86ff 9fdee1ed6a78476089ea061ece7ec69e - default default] Created volume successfully. >2018-08-29 05:39:27.742 257031 INFO cinder.volume.manager [req-8304118c-ee50-43ae-a74b-25343d270ddf 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Deleted volume successfully. >2018-08-29 05:39:28.120 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:39:28.238 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:39:29.532 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c9f80249-1a18-42ae-a1bf-545afc98b6ac 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Unable to get remote copy information for volume volume-4d4067c3-55ae-4d83-9c46-628d02423918 >2018-08-29 05:39:31.079 257256 INFO cinder.volume.flows.manager.create_volume [req-c9f80249-1a18-42ae-a1bf-545afc98b6ac 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Volume volume-4d4067c3-55ae-4d83-9c46-628d02423918 (4d4067c3-55ae-4d83-9c46-628d02423918): created successfully >2018-08-29 05:39:31.084 257256 INFO cinder.volume.manager [req-c9f80249-1a18-42ae-a1bf-545afc98b6ac 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Created volume successfully. >2018-08-29 05:39:33.059 257256 INFO cinder.volume.flows.manager.create_volume [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume b1a1686e-ab22-48c6-9bda-743cc717d9df: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-b1a1686e-ab22-48c6-9bda-743cc717d9df', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab5496b210>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:39:33.871 257256 INFO cinder.image.image_utils [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-29 05:39:34.414 257031 INFO cinder.volume.manager [req-ef744bf6-5ab7-4d1e-9e44-c0d785fd6cdc 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Deleted volume successfully. >2018-08-29 05:39:34.901 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 05:39:34.981 257031 INFO cinder.volume.manager [req-d57a6cfa-6cbc-4b95-b212-61441c36c233 f7d98a03c05f4120bc04f97eaecc86ff 9fdee1ed6a78476089ea061ece7ec69e - default default] Deleted volume successfully. >2018-08-29 05:39:35.006 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 05:39:35.108 257256 INFO cinder.volume.manager [req-f6ab4e4c-1c59-4811-a86c-33e11163e91d fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Deleted volume successfully. >2018-08-29 05:39:35.157 257256 INFO cinder.volume.manager [req-da66b1de-aed9-4825-abc6-519ce0ec48a7 fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Deleted volume successfully. >2018-08-29 05:39:35.926 257256 INFO cinder.volume.manager [req-7b609a32-e13c-4096-a215-c918c4457d2d fbdef2f4ff8246af98417b5b6044619a d93b33291fee4924bca4f8ea797ede7e - default default] Deleted volume successfully. >2018-08-29 05:39:36.885 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-98535e19-6422-466a-8566-a741b4ef8bcb 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:39:38.007 257256 INFO cinder.volume.manager [req-fdebbb4d-37f3-487e-bb9b-4632e14c3c7b 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Deleted volume successfully. >2018-08-29 05:39:38.499 257031 INFO cinder.volume.manager [req-98535e19-6422-466a-8566-a741b4ef8bcb 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Terminate volume connection completed successfully. >2018-08-29 05:39:40.392 257031 INFO cinder.volume.manager [req-4601c128-d61a-45e3-adb5-60ce4b3a98f1 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Terminate volume connection completed successfully. >2018-08-29 05:39:40.924 257031 INFO cinder.volume.manager [req-2cbce519-e725-49f7-a924-7d62eefe1628 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Detaching volume bb1daae0-ea69-413c-b52d-a93e806a27a2 from instance fd4efa61-941b-4112-88ba-f68eb7aa9804. >2018-08-29 05:39:41.011 257031 INFO cinder.volume.manager [req-2cbce519-e725-49f7-a924-7d62eefe1628 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Detach volume completed successfully. >2018-08-29 05:39:43.559 257256 WARNING cinder.db.sqlalchemy.api [req-06d1adcc-adce-4424-bd23-8604316befe7 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-29 05:39:44.972 257031 INFO cinder.volume.manager [req-004aea51-25d5-473a-8755-0d085e94f0f3 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Deleted volume successfully. >2018-08-29 05:39:45.493 257256 INFO cinder.volume.manager [req-06d1adcc-adce-4424-bd23-8604316befe7 370e7a8931ce4548baf458ddf8695e8b a2829c6e575b491a8a06d6ac41e0dc8b - default default] Deleted volume successfully. >2018-08-29 05:39:49.003 257256 INFO cinder.volume.manager [req-bafc04bc-9c65-422b-9e89-f63a27ec9aff 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:39:49.305 257031 INFO cinder.volume.manager [req-e7a59520-e105-472f-86f3-8cc50ce4a6f5 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Delete snapshot completed successfully. >2018-08-29 05:39:53.363 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-0d323ec5-65f1-4cc8-8dde-239d94a103ca 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] initialize_connection: Did not find a preferred node for volume volume-bb1daae0-ea69-413c-b52d-a93e806a27a2. >2018-08-29 05:39:53.620 257256 WARNING cinder.db.sqlalchemy.api [req-2a2b8916-ead3-4760-9ba1-bf92a9cc1051 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-29 05:39:54.360 257031 INFO cinder.volume.manager [req-0d323ec5-65f1-4cc8-8dde-239d94a103ca 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Initialize volume connection completed successfully. >2018-08-29 05:39:54.470 257031 INFO cinder.volume.manager [req-511712c2-81a1-4921-95a0-fb7510c1d2d4 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Deleted volume successfully. >2018-08-29 05:39:54.642 257256 INFO cinder.volume.manager [req-2a2b8916-ead3-4760-9ba1-bf92a9cc1051 3fc7394280fb45ed90832f6dad85bf2e 0fb42d7db7c7401b870633fd64557b25 - default default] Deleted volume successfully. >2018-08-29 05:39:55.058 257031 INFO cinder.volume.manager [req-b90b7e63-4d8b-449e-9a26-3c2aa66e4e5b 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Attaching volume bb1daae0-ea69-413c-b52d-a93e806a27a2 to instance fd4efa61-941b-4112-88ba-f68eb7aa9804 at mountpoint /dev/vdb on host None. >2018-08-29 05:39:55.145 257031 INFO cinder.volume.manager [req-b90b7e63-4d8b-449e-9a26-3c2aa66e4e5b 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Attach volume completed successfully. >2018-08-29 05:39:55.555 257256 INFO cinder.volume.manager [req-4816d90b-c21c-4eec-8a8c-7c73bf4014fc 0cd6539e214049e3b3c1ed685777b21a b787d472844040f4a9dc71eb3ae0d8b9 - default default] Group tempest-Group-506750657: creating >2018-08-29 05:39:55.604 257256 INFO cinder.volume.manager [req-4816d90b-c21c-4eec-8a8c-7c73bf4014fc 0cd6539e214049e3b3c1ed685777b21a b787d472844040f4a9dc71eb3ae0d8b9 - default default] Group tempest-Group-506750657: created successfully >2018-08-29 05:39:55.630 257256 INFO cinder.volume.manager [req-4816d90b-c21c-4eec-8a8c-7c73bf4014fc 0cd6539e214049e3b3c1ed685777b21a b787d472844040f4a9dc71eb3ae0d8b9 - default default] Create group completed successfully. >2018-08-29 05:39:58.328 257256 INFO cinder.volume.manager [req-5e9e7ee5-98c3-486c-95eb-fb98c0221f66 0cd6539e214049e3b3c1ed685777b21a b787d472844040f4a9dc71eb3ae0d8b9 - default default] Delete group completed successfully. >2018-08-29 05:40:03.205 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] initialize_connection: Did not find a preferred node for volume volume-b1a1686e-ab22-48c6-9bda-743cc717d9df. >2018-08-29 05:40:04.889 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:40:09.307 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:40:09.636 257256 INFO cinder.volume.flows.manager.create_volume [req-5f8ad90b-7def-4bed-8573-2c1c880eeb69 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] Volume bb8037ac-a774-4985-afb6-f065f57b64d9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bb8037ac-a774-4985-afb6-f065f57b64d9'} >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: b1a1686e-ab22-48c6-9bda-743cc717d9df: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:09.924 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:40:09.934 257256 WARNING cinder.volume.manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (409a25ca-2bf5-40c0-87a1-93ebf6b7e5e7) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:39:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1575638178',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=b1a1686e-ab22-48c6-9bda-743cc717d9df,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='cb3e0520906248498269661f5ea7d7f7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:39:32Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:39:32Z,user_id='1c84b2a696b64368a0e644491527ca7b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5433da90>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:39:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1575638178',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=b1a1686e-ab22-48c6-9bda-743cc717d9df,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='cb3e0520906248498269661f5ea7d7f7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:39:32Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:39:32Z,user_id='1c84b2a696b64368a0e644491527ca7b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b1a1686e-ab22-48c6-9bda-743cc717d9df),volume_id=b1a1686e-ab22-48c6-9bda-743cc717d9df,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab5433da90>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': 'b1a1686e-ab22-48c6-9bda-743cc717d9df', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab5496b210>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-b1a1686e-ab22-48c6-9bda-743cc717d9df', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:39:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1575638178',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=b1a1686e-ab22-48c6-9bda-743cc717d9df,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='cb3e0520906248498269661f5ea7d7f7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:39:32Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:39:32Z,user_id='1c84b2a696b64368a0e644491527ca7b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'1c84b2a696b64368a0e644491527ca7b', u'availability_zone': u'nova', u'reservations': [u'4ba25d79-a067-44e2-9a41-411f4966e624', u'0e850143-c66e-4cc5-a924-37de8608bd02'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {u'Type': u'Test'}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'cb3e0520906248498269661f5ea7d7f7', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesGetTest-Volume-1575638178'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'1c84b2a696b64368a0e644491527ca7b', u'availability_zone': u'nova', u'reservations': [u'4ba25d79-a067-44e2-9a41-411f4966e624', u'0e850143-c66e-4cc5-a924-37de8608bd02'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {u'Type': u'Test'}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'cb3e0520906248498269661f5ea7d7f7', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesGetTest-Volume-1575638178'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'b1a1686e-ab22-48c6-9bda-743cc717d9df', u'size': 1, u'display_name': u'tempest-VolumesGetTest-Volume-1575638178', u'display_description': None, u'cluster_name': None, u'name_id': u'b1a1686e-ab22-48c6-9bda-743cc717d9df', u'volume_admin_metadata': [], u'project_id': u'cb3e0520906248498269661f5ea7d7f7', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-b1a1686e-ab22-48c6-9bda-743cc717d9df', u'user_id': u'1c84b2a696b64368a0e644491527ca7b', u'bootable': False, u'created_at': u'2018-08-29T05:39:32.000000', u'attach_status': u'detached', u'volume_metadata': [{u'value': u'Test', u'key': u'Type'}], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {u'Type': u'Test'}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'b1a1686e-ab22-48c6-9bda-743cc717d9df', u'group_id': None}, u'user_id': u'1c84b2a696b64368a0e644491527ca7b', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {u'Type': u'Test'}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab5433da90>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b1a1686e-ab22-48c6-9bda-743cc717d9df),volume_id=b1a1686e-ab22-48c6-9bda-743cc717d9df,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:39:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1575638178',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=b1a1686e-ab22-48c6-9bda-743cc717d9df,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='cb3e0520906248498269661f5ea7d7f7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:39:32Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:39:32Z,user_id='1c84b2a696b64368a0e644491527ca7b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab5433da90>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:39:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1575638178',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=b1a1686e-ab22-48c6-9bda-743cc717d9df,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='cb3e0520906248498269661f5ea7d7f7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:39:32Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:39:32Z,user_id='1c84b2a696b64368a0e644491527ca7b',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:09.934 257256 ERROR cinder.volume.manager >2018-08-29 05:40:09.940 257256 WARNING cinder.volume.manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (409a25ca-2bf5-40c0-87a1-93ebf6b7e5e7) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:09.943 257256 WARNING cinder.volume.manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e5c2789a-79cf-48cf-aadf-3cb3949f024b) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:09.947 257256 WARNING cinder.volume.manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (24798184-ba0e-44a4-8746-a7413835cd17) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:09.957 257256 ERROR cinder.volume.flows.manager.create_volume [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Volume b1a1686e-ab22-48c6-9bda-743cc717d9df: create failed >2018-08-29 05:40:09.959 257256 WARNING cinder.volume.manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0f75a2fa-aa97-4f87-9da3-5911932b05bf) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:09.962 257256 WARNING cinder.volume.manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a28328b4-ba28-4fb6-a182-48d29b561364) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:09.964 257256 WARNING cinder.volume.manager [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Flow 'volume_create_manager' (60d339cd-aba0-4c5d-aa7b-b6bb4b504dbd) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server [req-e3c32f18-01cb-4b2d-85ec-5fd8c392b079 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:09.970 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:40:10.450 257031 INFO cinder.volume.flows.manager.create_volume [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Volume 5290ac13-7b40-4573-ade4-1584808e3071: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-5290ac13-7b40-4573-ade4-1584808e3071', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54504550>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:40:10.797 257256 INFO cinder.volume.flows.manager.create_volume [req-5f8ad90b-7def-4bed-8573-2c1c880eeb69 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] Volume volume-bb8037ac-a774-4985-afb6-f065f57b64d9 (bb8037ac-a774-4985-afb6-f065f57b64d9): created successfully >2018-08-29 05:40:10.801 257256 INFO cinder.volume.manager [req-5f8ad90b-7def-4bed-8573-2c1c880eeb69 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] Created volume successfully. >2018-08-29 05:40:11.655 257031 INFO cinder.image.image_utils [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Image download 12.00 MB at 9.97 MB/s >2018-08-29 05:40:14.163 257256 INFO cinder.volume.manager [req-a86c9408-fca9-463e-9dd6-b1976de5e5bb 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Deleted volume successfully. >2018-08-29 05:40:14.567 257256 INFO cinder.volume.manager [req-f010eb90-6d56-4546-a334-0be29c3e3102 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] Deleted volume successfully. >2018-08-29 05:40:14.938 257256 INFO cinder.volume.flows.manager.create_volume [req-23d5140d-195f-4044-abe2-fe69a533ef7a 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] Volume 3de3b18b-7bb4-493f-a496-b25d99be568c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3de3b18b-7bb4-493f-a496-b25d99be568c'} >2018-08-29 05:40:16.869 257256 INFO cinder.volume.flows.manager.create_volume [req-23d5140d-195f-4044-abe2-fe69a533ef7a 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] Volume volume-3de3b18b-7bb4-493f-a496-b25d99be568c (3de3b18b-7bb4-493f-a496-b25d99be568c): created successfully >2018-08-29 05:40:16.873 257256 INFO cinder.volume.manager [req-23d5140d-195f-4044-abe2-fe69a533ef7a 50dcd34971b14435a47bfbb1323d1652 0e71f744237e4d5fb80b4d87990987a0 - default default] Created volume successfully. >2018-08-29 05:40:17.827 257031 INFO cinder.volume.manager [req-652328b4-76b5-4a06-8a09-a4c49b702e03 1c84b2a696b64368a0e644491527ca7b cb3e0520906248498269661f5ea7d7f7 - default default] Deleted volume successfully. >2018-08-29 05:40:18.511 257256 INFO cinder.volume.manager [req-75fc183e-905a-4122-ae58-9d7c79582724 3ace57b8f00249ebbb6936379ac13d4d 484f39224a4f419f8741f2c05c45cd6a - default default] Transfer volume completed successfully. >2018-08-29 05:40:22.760 257256 INFO cinder.volume.manager [req-e11401e7-f599-446a-bc93-ded8625f0747 49d12061b1b641e88faa2d9985f746eb 444f5f590c0a4e07859dd80ae15ef5d7 - default default] Deleted volume successfully. >2018-08-29 05:40:25.540 257031 INFO cinder.volume.manager [req-9db621af-4c5c-4b8d-a4fe-be6bd79706e7 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Terminate volume connection completed successfully. >2018-08-29 05:40:26.482 257031 INFO cinder.volume.manager [req-beef3236-a1ba-42d0-b592-7399da2ea47a 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Detaching volume bb1daae0-ea69-413c-b52d-a93e806a27a2 from instance fd4efa61-941b-4112-88ba-f68eb7aa9804. >2018-08-29 05:40:26.565 257031 INFO cinder.volume.manager [req-beef3236-a1ba-42d0-b592-7399da2ea47a 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Detach volume completed successfully. >2018-08-29 05:40:31.405 257256 INFO cinder.volume.flows.manager.create_volume [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Volume 2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54415b90>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:40:33.042 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] initialize_connection: Did not find a preferred node for volume volume-5290ac13-7b40-4573-ade4-1584808e3071. >2018-08-29 05:40:35.168 257031 WARNING os_brick.initiator.connectors.fibre_channel [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:40:37.381 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-2efe79f6-edb6-4c4b-ae5c-6e6091b82a88 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] initialize_connection: Did not find a preferred node for volume volume-bb1daae0-ea69-413c-b52d-a93e806a27a2. >2018-08-29 05:40:37.986 257031 INFO cinder.volume.manager [req-2efe79f6-edb6-4c4b-ae5c-6e6091b82a88 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Initialize volume connection completed successfully. >2018-08-29 05:40:38.201 257031 INFO cinder.volume.manager [req-026ada99-611c-4f49-aa16-6c0d324846e0 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Attaching volume bb1daae0-ea69-413c-b52d-a93e806a27a2 to instance fd4efa61-941b-4112-88ba-f68eb7aa9804 at mountpoint /dev/vdb on host None. >2018-08-29 05:40:38.275 257031 INFO cinder.volume.manager [req-026ada99-611c-4f49-aa16-6c0d324846e0 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Attach volume completed successfully. >2018-08-29 05:40:39.909 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 5290ac13-7b40-4573-ade4-1584808e3071: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:40.621 257031 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:40:40.633 257031 WARNING cinder.volume.manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b5ea41b-f20c-411c-9ab6-f8844f909e1a) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeMigrateRetypeAttached-volume-379415179',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=5290ac13-7b40-4573-ade4-1584808e3071,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='871dccec6fa74720ad9da346f4144959',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:10Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:10Z,user_id='9dee4a5700914286bedcf4950f9a28c5',volume_attachment=<?>,volume_type=VolumeType(29154d47-b43f-4ba6-8b0e-936a9fc87ab1),volume_type_id=29154d47-b43f-4ba6-8b0e-936a9fc87ab1), 'context': <cinder.context.RequestContext object at 0x7fab53fc56d0>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeMigrateRetypeAttached-volume-379415179',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=5290ac13-7b40-4573-ade4-1584808e3071,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='871dccec6fa74720ad9da346f4144959',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:10Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:10Z,user_id='9dee4a5700914286bedcf4950f9a28c5',volume_attachment=<?>,volume_type=VolumeType(29154d47-b43f-4ba6-8b0e-936a9fc87ab1),volume_type_id=29154d47-b43f-4ba6-8b0e-936a9fc87ab1), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(5290ac13-7b40-4573-ade4-1584808e3071),volume_id=5290ac13-7b40-4573-ade4-1584808e3071,volume_properties=VolumeProperties,volume_type=VolumeType(29154d47-b43f-4ba6-8b0e-936a9fc87ab1)), 'context': <cinder.context.RequestContext object at 0x7fab53fc56d0>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '5290ac13-7b40-4573-ade4-1584808e3071', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54504550>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-5290ac13-7b40-4573-ade4-1584808e3071', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeMigrateRetypeAttached-volume-379415179',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=5290ac13-7b40-4573-ade4-1584808e3071,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='871dccec6fa74720ad9da346f4144959',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:10Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:10Z,user_id='9dee4a5700914286bedcf4950f9a28c5',volume_attachment=<?>,volume_type=VolumeType(29154d47-b43f-4ba6-8b0e-936a9fc87ab1),volume_type_id=29154d47-b43f-4ba6-8b0e-936a9fc87ab1), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': u'29154d47-b43f-4ba6-8b0e-936a9fc87ab1', u'group_id': None, u'user_id': u'9dee4a5700914286bedcf4950f9a28c5', u'availability_zone': u'nova', u'reservations': [u'6097feb5-2cc4-4cb8-968b-cd7cdb2d129e', u'3c800686-55d8-4d27-8a76-5330da1218b0', u'85e4b168-dc44-4acc-8ab8-600cd92fecfd', u'705b2f76-b642-4bb4-8948-29c0e5005950'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'871dccec6fa74720ad9da346f4144959', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestVolumeMigrateRetypeAttached-volume-379415179'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': u'29154d47-b43f-4ba6-8b0e-936a9fc87ab1', u'group_id': None, u'user_id': u'9dee4a5700914286bedcf4950f9a28c5', u'availability_zone': u'nova', u'reservations': [u'6097feb5-2cc4-4cb8-968b-cd7cdb2d129e', u'3c800686-55d8-4d27-8a76-5330da1218b0', u'85e4b168-dc44-4acc-8ab8-600cd92fecfd', u'705b2f76-b642-4bb4-8948-29c0e5005950'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'871dccec6fa74720ad9da346f4144959', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestVolumeMigrateRetypeAttached-volume-379415179'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'5290ac13-7b40-4573-ade4-1584808e3071', u'size': 1, u'display_name': u'tempest-TestVolumeMigrateRetypeAttached-volume-379415179', u'display_description': None, u'cluster_name': None, u'name_id': u'5290ac13-7b40-4573-ade4-1584808e3071', u'volume_admin_metadata': [], u'project_id': u'871dccec6fa74720ad9da346f4144959', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': u'29154d47-b43f-4ba6-8b0e-936a9fc87ab1', u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-5290ac13-7b40-4573-ade4-1584808e3071', u'user_id': u'9dee4a5700914286bedcf4950f9a28c5', u'bootable': False, u'created_at': u'2018-08-29T05:40:10.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': {u'name': u'tempest-scenario-type-tempest-TestVolumeMigrateRetypeAttached-volume-type-2054444895-106595907', u'qos_specs_id': None, u'deleted': False, u'created_at': u'2018-08-29T05:40:08.000000', u'updated_at': None, u'extra_specs': {u'volume_backend_name': u'svc1'}, u'is_public': True, u'deleted_at': None, u'id': u'29154d47-b43f-4ba6-8b0e-936a9fc87ab1', u'projects': [], u'description': None}, u'volume_id': u'5290ac13-7b40-4573-ade4-1584808e3071', u'group_id': None}, u'user_id': u'9dee4a5700914286bedcf4950f9a28c5', u'availability_zone': u'nova', u'volume_type': VolumeType(created_at=2018-08-29T05:40:08Z,deleted=False,deleted_at=None,description=None,extra_specs={volume_backend_name='svc1'},id=29154d47-b43f-4ba6-8b0e-936a9fc87ab1,is_public=True,name='tempest-scenario-type-tempest-TestVolumeMigrateRetypeAttached-volume-type-2054444895-106595907',projects=[],qos_specs=<?>,qos_specs_id=None,updated_at=None), u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc1#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': VolumeType(created_at=2018-08-29T05:40:08Z,deleted=False,deleted_at=None,description=None,extra_specs={volume_backend_name='svc1'},id=29154d47-b43f-4ba6-8b0e-936a9fc87ab1,is_public=True,name='tempest-scenario-type-tempest-TestVolumeMigrateRetypeAttached-volume-type-2054444895-106595907',projects=[],qos_specs=<?>,qos_specs_id=None,updated_at=None), u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab53fc56d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(5290ac13-7b40-4573-ade4-1584808e3071),volume_id=5290ac13-7b40-4573-ade4-1584808e3071,volume_properties=VolumeProperties,volume_type=VolumeType(29154d47-b43f-4ba6-8b0e-936a9fc87ab1))}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeMigrateRetypeAttached-volume-379415179',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=5290ac13-7b40-4573-ade4-1584808e3071,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='871dccec6fa74720ad9da346f4144959',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:10Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:10Z,user_id='9dee4a5700914286bedcf4950f9a28c5',volume_attachment=<?>,volume_type=VolumeType(29154d47-b43f-4ba6-8b0e-936a9fc87ab1),volume_type_id=29154d47-b43f-4ba6-8b0e-936a9fc87ab1), 'context': <cinder.context.RequestContext object at 0x7fab53fc56d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeMigrateRetypeAttached-volume-379415179',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc1#NonDR_DS8KBH00_Pool',id=5290ac13-7b40-4573-ade4-1584808e3071,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='871dccec6fa74720ad9da346f4144959',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:10Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:10Z,user_id='9dee4a5700914286bedcf4950f9a28c5',volume_attachment=<?>,volume_type=VolumeType(29154d47-b43f-4ba6-8b0e-936a9fc87ab1),volume_type_id=29154d47-b43f-4ba6-8b0e-936a9fc87ab1)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager image_service) >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager image_service >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:40.633 257031 ERROR cinder.volume.manager >2018-08-29 05:40:40.639 257031 WARNING cinder.volume.manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b5ea41b-f20c-411c-9ab6-f8844f909e1a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:40.643 257031 WARNING cinder.volume.manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2d1070de-93a0-4f74-b987-70b3df12e90f) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:40.646 257031 WARNING cinder.volume.manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (308308c5-006c-481f-858a-edef2b395a0e) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:40.653 257031 ERROR cinder.volume.flows.manager.create_volume [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Volume 5290ac13-7b40-4573-ade4-1584808e3071: create failed >2018-08-29 05:40:40.655 257031 WARNING cinder.volume.manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6575743d-66e2-458c-a9ed-144b818cecd8) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:40.658 257031 WARNING cinder.volume.manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (14276350-596f-4483-8e6c-aec8ac1e8b5e) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:40:40.661 257031 WARNING cinder.volume.manager [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Flow 'volume_create_manager' (a4a1bf16-14c3-4355-802d-b76c86e02ba7) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server [req-931614fb-daea-48f0-aa94-875cac333459 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:40:40.666 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:40:41.979 257256 INFO cinder.image.image_utils [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Image download 12.00 MB at 9.16 MB/s >2018-08-29 05:40:43.503 257031 WARNING cinder.quota [req-d1205093-3f2a-4814-8491-160225d208e5 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Deprecated: Default quota for resource: volumes_tempest-scenario-type-tempest-TestVolumeMigrateRetypeAttached-volume-type-2054444895-106595907 is set by the default quota flag: quota_volumes_tempest-scenario-type-tempest-TestVolumeMigrateRetypeAttached-volume-type-2054444895-106595907, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:40:43.504 257031 WARNING cinder.quota [req-d1205093-3f2a-4814-8491-160225d208e5 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Deprecated: Default quota for resource: gigabytes_tempest-scenario-type-tempest-TestVolumeMigrateRetypeAttached-volume-type-2054444895-106595907 is set by the default quota flag: quota_gigabytes_tempest-scenario-type-tempest-TestVolumeMigrateRetypeAttached-volume-type-2054444895-106595907, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:40:44.709 257031 INFO cinder.volume.manager [req-d1205093-3f2a-4814-8491-160225d208e5 9dee4a5700914286bedcf4950f9a28c5 871dccec6fa74720ad9da346f4144959 - default default] Deleted volume successfully. >2018-08-29 05:41:00.193 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] initialize_connection: Did not find a preferred node for volume volume-2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b. >2018-08-29 05:41:02.032 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:41:03.154 257031 INFO cinder.volume.manager [req-af8b8794-063b-4544-9eb5-d7723e737db7 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Terminate volume connection completed successfully. >2018-08-29 05:41:03.323 257031 INFO cinder.volume.manager [req-2798430e-defa-4446-9cf0-976e5e4135da 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Detaching volume bb1daae0-ea69-413c-b52d-a93e806a27a2 from instance fd4efa61-941b-4112-88ba-f68eb7aa9804. >2018-08-29 05:41:03.402 257031 INFO cinder.volume.manager [req-2798430e-defa-4446-9cf0-976e5e4135da 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Detach volume completed successfully. >2018-08-29 05:41:06.492 257256 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Need to remove FC Zone, building initiator target map. >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:41:07.051 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:41:07.063 257256 WARNING cinder.volume.manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (11489286-a361-4e6e-a96f-08a6696c0d42) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-152065012',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ba7d06c91116485180b1ec55ae827a2e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:31Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:31Z,user_id='2b7697817fde4913a0b34f53e45778e0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54909050>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-152065012',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ba7d06c91116485180b1ec55ae827a2e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:31Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:31Z,user_id='2b7697817fde4913a0b34f53e45778e0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b),volume_id=2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab54909050>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab54415b90>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-152065012',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ba7d06c91116485180b1ec55ae827a2e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:31Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:31Z,user_id='2b7697817fde4913a0b34f53e45778e0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'2b7697817fde4913a0b34f53e45778e0', u'availability_zone': u'nova', u'reservations': [u'053fb63d-0ab5-4a48-bef5-b5ace2a5b966', u'95580ba4-6fc0-4349-8d41-cebac768bd64'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'ba7d06c91116485180b1ec55ae827a2e', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesCloneTest-Volume-152065012'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'2b7697817fde4913a0b34f53e45778e0', u'availability_zone': u'nova', u'reservations': [u'053fb63d-0ab5-4a48-bef5-b5ace2a5b966', u'95580ba4-6fc0-4349-8d41-cebac768bd64'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'ba7d06c91116485180b1ec55ae827a2e', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesCloneTest-Volume-152065012'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b', u'size': 1, u'display_name': u'tempest-VolumesCloneTest-Volume-152065012', u'display_description': None, u'cluster_name': None, u'name_id': u'2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b', u'volume_admin_metadata': [], u'project_id': u'ba7d06c91116485180b1ec55ae827a2e', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b', u'user_id': u'2b7697817fde4913a0b34f53e45778e0', u'bootable': False, u'created_at': u'2018-08-29T05:40:30.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b', u'group_id': None}, u'user_id': u'2b7697817fde4913a0b34f53e45778e0', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab54909050>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b),volume_id=2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-152065012',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ba7d06c91116485180b1ec55ae827a2e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:31Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:31Z,user_id='2b7697817fde4913a0b34f53e45778e0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54909050>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:40:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-152065012',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ba7d06c91116485180b1ec55ae827a2e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:40:31Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:40:31Z,user_id='2b7697817fde4913a0b34f53e45778e0',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:41:07.063 257256 ERROR cinder.volume.manager >2018-08-29 05:41:07.069 257256 WARNING cinder.volume.manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (11489286-a361-4e6e-a96f-08a6696c0d42) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:41:07.072 257256 WARNING cinder.volume.manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d2cea0ea-a092-4973-b8a9-785c3b85b049) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:41:07.075 257256 WARNING cinder.volume.manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6c33f29f-0c7b-4b96-90e1-5463b30004e9) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:41:07.082 257256 ERROR cinder.volume.flows.manager.create_volume [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Volume 2b92e0f9-23d1-4bd6-9607-bfb62fd96d1b: create failed >2018-08-29 05:41:07.084 257256 WARNING cinder.volume.manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (415ac418-e742-4495-98e5-e3d4eff81074) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:41:07.087 257256 WARNING cinder.volume.manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (52fcc191-b56a-4588-855c-4c02ddb3bad2) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:41:07.089 257256 WARNING cinder.volume.manager [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Flow 'volume_create_manager' (bef79d80-fe30-44ae-82bc-cc6dc8eaf457) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server [req-ced10291-ea58-485a-bb59-8915fd79e585 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:41:07.093 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:41:07.645 257031 INFO cinder.volume.flows.manager.create_volume [req-00908133-865c-4ed4-8d7a-6faebf4026b4 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Volume f6130ac5-0117-4683-8970-d9a6d1303dcc: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f6130ac5-0117-4683-8970-d9a6d1303dcc'} >2018-08-29 05:41:08.692 257031 INFO cinder.volume.flows.manager.create_volume [req-00908133-865c-4ed4-8d7a-6faebf4026b4 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Volume volume-f6130ac5-0117-4683-8970-d9a6d1303dcc (f6130ac5-0117-4683-8970-d9a6d1303dcc): created successfully >2018-08-29 05:41:08.696 257031 INFO cinder.volume.manager [req-00908133-865c-4ed4-8d7a-6faebf4026b4 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Created volume successfully. >2018-08-29 05:41:10.305 257031 INFO cinder.volume.flows.manager.create_volume [req-c224adde-1717-4a88-a9bd-1fff50c2ba89 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Volume b5d6480d-8a2e-4287-abd4-935bf29c8cad: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-b5d6480d-8a2e-4287-abd4-935bf29c8cad', 'source_volid': 'f6130ac5-0117-4683-8970-d9a6d1303dcc', 'source_volstatus': u'available'} >2018-08-29 05:41:28.459 257031 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c224adde-1717-4a88-a9bd-1fff50c2ba89 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Unable to get remote copy information for volume volume-b5d6480d-8a2e-4287-abd4-935bf29c8cad >2018-08-29 05:41:29.009 257031 INFO cinder.volume.flows.manager.create_volume [req-c224adde-1717-4a88-a9bd-1fff50c2ba89 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Volume volume-b5d6480d-8a2e-4287-abd4-935bf29c8cad (b5d6480d-8a2e-4287-abd4-935bf29c8cad): created successfully >2018-08-29 05:41:29.014 257031 INFO cinder.volume.manager [req-c224adde-1717-4a88-a9bd-1fff50c2ba89 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Created volume successfully. >2018-08-29 05:41:32.876 257256 INFO cinder.volume.manager [req-2be02c2e-fe14-4065-91d0-c6f1596bdaf7 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Deleted volume successfully. >2018-08-29 05:41:33.045 257031 INFO cinder.volume.manager [req-55a2a0ff-d19c-44be-9c51-0f19312459c3 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Deleted volume successfully. >2018-08-29 05:41:33.320 257031 INFO cinder.volume.manager [req-775b5764-c104-4816-9c48-c5d722a0b6fd 2b7697817fde4913a0b34f53e45778e0 ba7d06c91116485180b1ec55ae827a2e - default default] Deleted volume successfully. >2018-08-29 05:42:15.999 257031 INFO cinder.volume.manager [req-ff175d5b-2d6d-4f23-8afd-8c22bd553521 8017d35454b94b9ab3c069e6ba1869c0 dbdd5434b35349e7a3a55237450b3175 - default default] Deleted volume successfully. >2018-08-29 05:42:36.541 257031 INFO cinder.volume.flows.manager.create_volume [req-f9eb6d45-7998-4b1f-af43-0b3a68c88c4d 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Volume 078b107e-ec86-4be1-92ad-249bec798b16: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-078b107e-ec86-4be1-92ad-249bec798b16'} >2018-08-29 05:42:37.271 257031 INFO cinder.volume.flows.manager.create_volume [req-f9eb6d45-7998-4b1f-af43-0b3a68c88c4d 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Volume volume-078b107e-ec86-4be1-92ad-249bec798b16 (078b107e-ec86-4be1-92ad-249bec798b16): created successfully >2018-08-29 05:42:37.274 257031 INFO cinder.volume.manager [req-f9eb6d45-7998-4b1f-af43-0b3a68c88c4d 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Created volume successfully. >2018-08-29 05:42:39.125 257256 INFO cinder.volume.flows.manager.create_volume [req-41efbaa5-178f-4b69-b05b-600153a7e506 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Volume f0bd2b62-2bd4-46e4-9109-9bffa1065297: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f0bd2b62-2bd4-46e4-9109-9bffa1065297'} >2018-08-29 05:42:39.754 257256 INFO cinder.volume.flows.manager.create_volume [req-41efbaa5-178f-4b69-b05b-600153a7e506 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Volume volume-f0bd2b62-2bd4-46e4-9109-9bffa1065297 (f0bd2b62-2bd4-46e4-9109-9bffa1065297): created successfully >2018-08-29 05:42:39.758 257256 INFO cinder.volume.manager [req-41efbaa5-178f-4b69-b05b-600153a7e506 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Created volume successfully. >2018-08-29 05:42:41.524 257031 INFO cinder.volume.flows.manager.create_volume [req-493b9118-1c80-45a9-b4e6-3709a7e74ae0 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Volume d54fcf6f-1dab-47ea-aa0e-b8be7630a1d2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d54fcf6f-1dab-47ea-aa0e-b8be7630a1d2'} >2018-08-29 05:42:42.174 257031 INFO cinder.volume.flows.manager.create_volume [req-493b9118-1c80-45a9-b4e6-3709a7e74ae0 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Volume volume-d54fcf6f-1dab-47ea-aa0e-b8be7630a1d2 (d54fcf6f-1dab-47ea-aa0e-b8be7630a1d2): created successfully >2018-08-29 05:42:42.178 257031 INFO cinder.volume.manager [req-493b9118-1c80-45a9-b4e6-3709a7e74ae0 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Created volume successfully. >2018-08-29 05:42:47.773 257031 INFO cinder.volume.manager [req-15353938-709d-4309-a6e0-884684e3dce8 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Deleted volume successfully. >2018-08-29 05:42:49.145 257256 INFO cinder.volume.manager [req-1023b336-ea14-48cd-aed0-452dced4b1f4 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Deleted volume successfully. >2018-08-29 05:42:50.957 257031 INFO cinder.volume.manager [req-252ac786-8e17-4707-b471-34ce22665e52 89d970a5c3d74575b40ede98fdd3c681 45e5a55125174907b9f8b93da0d09702 - default default] Deleted volume successfully. >2018-08-29 05:43:28.061 257031 INFO cinder.volume.flows.manager.create_volume [req-64c2e103-dc8b-4ba6-88f5-de6ecd04d69f b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Volume 00da85bd-20c2-4400-b7f4-d6775432eab4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-00da85bd-20c2-4400-b7f4-d6775432eab4'} >2018-08-29 05:43:28.621 257031 INFO cinder.volume.flows.manager.create_volume [req-64c2e103-dc8b-4ba6-88f5-de6ecd04d69f b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Volume volume-00da85bd-20c2-4400-b7f4-d6775432eab4 (00da85bd-20c2-4400-b7f4-d6775432eab4): created successfully >2018-08-29 05:43:28.625 257031 INFO cinder.volume.manager [req-64c2e103-dc8b-4ba6-88f5-de6ecd04d69f b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Created volume successfully. >2018-08-29 05:43:30.525 257256 INFO cinder.volume.flows.manager.create_volume [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Volume 0d6ad1cc-bc88-465e-bb59-26334b4ff4e3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3'} >2018-08-29 05:43:31.092 257256 INFO cinder.volume.flows.manager.create_volume [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Volume volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3 (0d6ad1cc-bc88-465e-bb59-26334b4ff4e3): created successfully >2018-08-29 05:43:31.095 257256 INFO cinder.volume.manager [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Created volume successfully. >2018-08-29 05:43:39.245 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] initialize_connection: Did not find a preferred node for volume volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3. >2018-08-29 05:43:39.872 257256 INFO cinder.volume.manager [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Initialize volume connection completed successfully. >2018-08-29 05:43:40.264 257031 WARNING os_brick.initiator.connectors.fibre_channel [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Failed to copy volume 00da85bd-20c2-4400-b7f4-d6775432eab4 to 0d6ad1cc-bc88-465e-bb59-26334b4ff4e3: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2085, in _migrate_volume_generic >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager self._copy_volume_data(ctxt, volume, new_volume, remote='dest') >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1967, in _copy_volume_data >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager attach_encryptor=attach_encryptor) >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1901, in _attach_volume >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager attach_info = self._connect_device(conn) >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1865, in _connect_device >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager vol_handle = connector.connect_volume(conn['data']) >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager return f(*args, **kwargs) >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager return f(*args, **kwargs) >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager raise exception.NoFibreChannelHostsFound() >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:43:40.265 257031 ERROR cinder.volume.manager >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Exception during message handling: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2745, in retype >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server new_reservations, status_update) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2741, in retype >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server new_type_id=new_type_id) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2348, in migrate_volume >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server volume.save() >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2341, in migrate_volume >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server self._migrate_volume_generic(ctxt, volume, host, new_type_id) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2107, in _migrate_volume_generic >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server new_volume) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2085, in _migrate_volume_generic >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server self._copy_volume_data(ctxt, volume, new_volume, remote='dest') >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1967, in _copy_volume_data >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server attach_encryptor=attach_encryptor) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1901, in _attach_volume >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server attach_info = self._connect_device(conn) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1865, in _connect_device >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server vol_handle = connector.connect_volume(conn['data']) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server raise exception.NoFibreChannelHostsFound() >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:43:40.309 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:43:41.007 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Error has occurred: Unexpected error while running command. >Command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-29 05:43:44.249 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Error running SSH command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3": ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-29 05:43:44.250 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-29 05:43:44.251 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Error running SSH command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3": ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-29 05:43:44.252 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] CLI Exception output: > command: ['svctask', 'rmvdisk', '"volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3"'] > stdout: > stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-29 05:43:44.262 257256 ERROR cinder.volume.manager [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Unable to delete the destination volume during volume migration, (NOTE: database record needs to be deleted).: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'rmvdisk', '"volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3"'] > stdout: > stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >. >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server [req-b1075625-37de-4b74-85ff-ff88dc7a831a b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'rmvdisk', '"volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3"'] > stdout: > stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >. >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 793, in delete_volume >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server new_status) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 775, in delete_volume >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2556, in delete_volume >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server self._helpers.delete_vdisk(volume['name'], False) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1857, in delete_vdisk >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server self.ssh.rmvdisk(vdisk, force=force) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 458, in rmvdisk >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server self.run_ssh_assert_no_output(ssh_cmd) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 168, in run_ssh_assert_no_output >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server out, err = self._run_ssh(ssh_cmd) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=msg) >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server command: ['svctask', 'rmvdisk', '"volume-0d6ad1cc-bc88-465e-bb59-26334b4ff4e3"'] >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server stdout: >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server . >2018-08-29 05:43:44.263 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:48:31.503 257031 WARNING cinder.quota [req-e75ce7f4-3d24-405e-a148-d5f68555b28c b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-804286575 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-804286575, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:48:31.506 257031 WARNING cinder.quota [req-e75ce7f4-3d24-405e-a148-d5f68555b28c b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-804286575 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-804286575, it is now deprecated. Please use the default quota class for default quota. >2018-08-29 05:48:31.791 257031 INFO cinder.volume.manager [req-e75ce7f4-3d24-405e-a148-d5f68555b28c b6e13b1064a343a2943617244daa74d2 f734b210bf654e578c844f62e192c9a6 - default default] Deleted volume successfully. >2018-08-29 05:48:38.032 257031 INFO cinder.volume.flows.manager.create_volume [req-2544c0a8-0081-434d-b4b8-8ace35dda104 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Volume 49d5777e-0630-404e-874a-74f3485e6f31: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-49d5777e-0630-404e-874a-74f3485e6f31'} >2018-08-29 05:48:38.691 257031 INFO cinder.volume.flows.manager.create_volume [req-2544c0a8-0081-434d-b4b8-8ace35dda104 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Volume volume-49d5777e-0630-404e-874a-74f3485e6f31 (49d5777e-0630-404e-874a-74f3485e6f31): created successfully >2018-08-29 05:48:38.695 257031 INFO cinder.volume.manager [req-2544c0a8-0081-434d-b4b8-8ace35dda104 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Created volume successfully. >2018-08-29 05:48:45.876 257031 INFO cinder.volume.manager [req-bac39894-4229-466b-abdd-0e01e60c3ca4 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Attaching volume 49d5777e-0630-404e-874a-74f3485e6f31 to instance 9bc675a6-4c37-4557-bf4e-12ea11b44f8f at mountpoint /dev/vdb on host None. >2018-08-29 05:48:45.949 257031 INFO cinder.volume.manager [req-bac39894-4229-466b-abdd-0e01e60c3ca4 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Attach volume completed successfully. >2018-08-29 05:48:46.678 257031 INFO cinder.volume.manager [req-8b91830b-7981-46b9-ad57-782e9e3925fb 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Detaching volume 49d5777e-0630-404e-874a-74f3485e6f31 from instance 9bc675a6-4c37-4557-bf4e-12ea11b44f8f. >2018-08-29 05:48:46.756 257031 INFO cinder.volume.manager [req-8b91830b-7981-46b9-ad57-782e9e3925fb 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Detach volume completed successfully. >2018-08-29 05:48:53.091 257031 INFO cinder.volume.manager [req-ce247f58-9c50-40b7-a080-1df2b5f0c63f 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Attaching volume 49d5777e-0630-404e-874a-74f3485e6f31 to instance 5da81769-2329-472e-b1b3-403d21b99b42 at mountpoint /dev/vdb on host None. >2018-08-29 05:48:53.162 257031 INFO cinder.volume.manager [req-ce247f58-9c50-40b7-a080-1df2b5f0c63f 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Attach volume completed successfully. >2018-08-29 05:48:53.899 257031 INFO cinder.volume.manager [req-9b146505-23c5-4db3-b533-ec6f6fd5c697 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Detaching volume 49d5777e-0630-404e-874a-74f3485e6f31 from instance 5da81769-2329-472e-b1b3-403d21b99b42. >2018-08-29 05:48:54.300 257031 INFO cinder.volume.manager [req-9b146505-23c5-4db3-b533-ec6f6fd5c697 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Detach volume completed successfully. >2018-08-29 05:49:00.448 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-10b933c9-3d40-49c1-b8f6-041a642ebcb9 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] initialize_connection: Did not find a preferred node for volume volume-49d5777e-0630-404e-874a-74f3485e6f31. >2018-08-29 05:49:01.445 257031 WARNING os_brick.initiator.connectors.fibre_channel [req-10b933c9-3d40-49c1-b8f6-041a642ebcb9 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:49:03.278 257031 ERROR cinder.volume.manager [req-10b933c9-3d40-49c1-b8f6-041a642ebcb9 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Upload volume to image encountered an error (image-id: 5e2839a2-68fd-43db-b8c2-80fb664535d4).: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:49:03.280 257031 INFO cinder.message.api [req-10b933c9-3d40-49c1-b8f6-041a642ebcb9 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Creating message record for request_id = req-10b933c9-3d40-49c1-b8f6-041a642ebcb9 >2018-08-29 05:49:03.559 257031 WARNING cinder.volume.manager [req-10b933c9-3d40-49c1-b8f6-041a642ebcb9 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Deleting image in unexpected status: queued.: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server [req-10b933c9-3d40-49c1-b8f6-041a642ebcb9 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Exception during message handling: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1522, in copy_volume_to_image >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server payload['message'] = six.text_type(error) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1503, in copy_volume_to_image >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server image_service, image_meta) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 850, in copy_volume_to_image >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server attach_info = self._connect_device(conn) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server device = connector.connect_volume(conn['data']) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server raise exception.NoFibreChannelHostsFound() >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:49:03.869 257031 ERROR oslo_messaging.rpc.server >2018-08-29 05:49:05.384 257031 INFO cinder.volume.manager [req-2b064098-a649-49c6-a9fc-cb31bf8fb12d 30cfe6c157c04ab0bcc5c05116c4208a 87bf6327dd5d4189bfda1952c047d1c9 - default default] Deleted volume successfully. >2018-08-29 05:49:29.754 257031 INFO cinder.volume.flows.manager.create_volume [req-88c5bd81-c4f6-47bd-8b7b-4944b16bbb44 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Volume e215809b-17c7-4f55-a533-faaa4177c25e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e215809b-17c7-4f55-a533-faaa4177c25e'} >2018-08-29 05:49:30.310 257031 INFO cinder.volume.flows.manager.create_volume [req-88c5bd81-c4f6-47bd-8b7b-4944b16bbb44 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Volume volume-e215809b-17c7-4f55-a533-faaa4177c25e (e215809b-17c7-4f55-a533-faaa4177c25e): created successfully >2018-08-29 05:49:30.315 257031 INFO cinder.volume.manager [req-88c5bd81-c4f6-47bd-8b7b-4944b16bbb44 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Created volume successfully. >2018-08-29 05:49:40.429 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-136f8539-4e3e-43f2-9d4f-51ac49c70ad8 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] initialize_connection: Did not find a preferred node for volume volume-e215809b-17c7-4f55-a533-faaa4177c25e. >2018-08-29 05:49:40.654 257031 INFO cinder.volume.manager [req-136f8539-4e3e-43f2-9d4f-51ac49c70ad8 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Initialize volume connection completed successfully. >2018-08-29 05:49:41.108 257031 INFO cinder.volume.manager [req-8a43e050-f44c-4784-8035-1f6bdb639d32 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Attaching volume e215809b-17c7-4f55-a533-faaa4177c25e to instance ec7b97ab-4dfc-40a9-b16a-d2ab15d9c646 at mountpoint /dev/vda on host None. >2018-08-29 05:49:41.192 257031 INFO cinder.volume.manager [req-8a43e050-f44c-4784-8035-1f6bdb639d32 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Attach volume completed successfully. >2018-08-29 05:54:50.980 257256 INFO cinder.volume.flows.manager.create_volume [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Volume 8ec1ef21-9fad-4c12-9f9d-b6d088ffe465: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-8ec1ef21-9fad-4c12-9f9d-b6d088ffe465', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab542b5c50>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:54:52.078 257256 INFO cinder.image.image_utils [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Image download 12.00 MB at 10.96 MB/s >2018-08-29 05:54:56.324 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] initialize_connection: Did not find a preferred node for volume volume-8ec1ef21-9fad-4c12-9f9d-b6d088ffe465. >2018-08-29 05:54:57.323 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 8ec1ef21-9fad-4c12-9f9d-b6d088ffe465: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:54:58.999 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:54:59.009 257256 WARNING cinder.volume.manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86cfa4da-0d17-455b-980c-719e7e79e89f) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:54:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1117370397',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=8ec1ef21-9fad-4c12-9f9d-b6d088ffe465,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:54:50Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:54:50Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab618f0a50>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:54:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1117370397',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=8ec1ef21-9fad-4c12-9f9d-b6d088ffe465,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:54:50Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:54:50Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(8ec1ef21-9fad-4c12-9f9d-b6d088ffe465),volume_id=8ec1ef21-9fad-4c12-9f9d-b6d088ffe465,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab618f0a50>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '8ec1ef21-9fad-4c12-9f9d-b6d088ffe465', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab542b5c50>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-8ec1ef21-9fad-4c12-9f9d-b6d088ffe465', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:54:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1117370397',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=8ec1ef21-9fad-4c12-9f9d-b6d088ffe465,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:54:50Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:54:50Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'availability_zone': u'nova', u'reservations': [u'3ea3692a-eaa0-4567-91b9-1493d8e6380c', u'7901f520-1e97-4a46-abed-c16564811ed4'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'065c0797e3cf47d08cced1bc93b84eb1', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestVolumeBootPattern-volume-origin-1117370397'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'availability_zone': u'nova', u'reservations': [u'3ea3692a-eaa0-4567-91b9-1493d8e6380c', u'7901f520-1e97-4a46-abed-c16564811ed4'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'065c0797e3cf47d08cced1bc93b84eb1', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestVolumeBootPattern-volume-origin-1117370397'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'8ec1ef21-9fad-4c12-9f9d-b6d088ffe465', u'size': 1, u'display_name': u'tempest-TestVolumeBootPattern-volume-origin-1117370397', u'display_description': None, u'cluster_name': None, u'name_id': u'8ec1ef21-9fad-4c12-9f9d-b6d088ffe465', u'volume_admin_metadata': [], u'project_id': u'065c0797e3cf47d08cced1bc93b84eb1', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-8ec1ef21-9fad-4c12-9f9d-b6d088ffe465', u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'bootable': False, u'created_at': u'2018-08-29T05:54:50.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'8ec1ef21-9fad-4c12-9f9d-b6d088ffe465', u'group_id': None}, u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab618f0a50>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(8ec1ef21-9fad-4c12-9f9d-b6d088ffe465),volume_id=8ec1ef21-9fad-4c12-9f9d-b6d088ffe465,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:54:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1117370397',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=8ec1ef21-9fad-4c12-9f9d-b6d088ffe465,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:54:50Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:54:50Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab618f0a50>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:54:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1117370397',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=8ec1ef21-9fad-4c12-9f9d-b6d088ffe465,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:54:50Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:54:50Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:54:59.009 257256 ERROR cinder.volume.manager >2018-08-29 05:54:59.015 257256 WARNING cinder.volume.manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86cfa4da-0d17-455b-980c-719e7e79e89f) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:54:59.018 257256 WARNING cinder.volume.manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (eb4aaadf-b1f9-408f-bc2b-d24fc258d3e9) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:54:59.021 257256 WARNING cinder.volume.manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cb15dc31-9390-451c-be9c-86b64cc9c14a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:54:59.028 257256 ERROR cinder.volume.flows.manager.create_volume [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Volume 8ec1ef21-9fad-4c12-9f9d-b6d088ffe465: create failed >2018-08-29 05:54:59.029 257256 WARNING cinder.volume.manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (15f22899-fedf-4715-a669-78837441449a) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:54:59.032 257256 WARNING cinder.volume.manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (af0ff7ac-09ae-4992-80f8-0491d4091fef) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:54:59.035 257256 WARNING cinder.volume.manager [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Flow 'volume_create_manager' (387e2887-dc6b-4433-a6e9-2ac9b5f42a94) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server [req-292ac8b0-cef0-4a0e-af7a-8215ef480d2a aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:54:59.039 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:55:01.130 257256 INFO cinder.volume.manager [req-df5d3791-bcdf-44b8-8031-ff23ad0c6626 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Deleted volume successfully. >2018-08-29 05:55:02.014 257256 INFO cinder.volume.flows.manager.create_volume [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Volume 28c651ab-31a5-452b-9583-c1d51f49dfcc: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-28c651ab-31a5-452b-9583-c1d51f49dfcc', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab547695d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-29 05:55:03.303 257256 INFO cinder.image.image_utils [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Image download 12.00 MB at 9.33 MB/s >2018-08-29 05:55:07.385 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] initialize_connection: Did not find a preferred node for volume volume-28c651ab-31a5-452b-9583-c1d51f49dfcc. >2018-08-29 05:55:08.384 257256 WARNING os_brick.initiator.connectors.fibre_channel [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] We are unable to locate any Fibre Channel devices >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 28c651ab-31a5-452b-9583-c1d51f49dfcc: NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 136, in connect_volume >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume raise exception.NoFibreChannelHostsFound() >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume NoFibreChannelHostsFound: We are unable to locate any Fibre Channel devices. >2018-08-29 05:55:10.332 257256 ERROR cinder.volume.flows.manager.create_volume >2018-08-29 05:55:10.343 257256 WARNING cinder.volume.manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68a7a052-6865-433a-b377-4ddd2b87cfa9) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:55:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-121709470',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=28c651ab-31a5-452b-9583-c1d51f49dfcc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:55:01Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:55:01Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54b7d7d0>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:55:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-121709470',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=28c651ab-31a5-452b-9583-c1d51f49dfcc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:55:01Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:55:01Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(28c651ab-31a5-452b-9583-c1d51f49dfcc),volume_id=28c651ab-31a5-452b-9583-c1d51f49dfcc,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fab54b7d7d0>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '28c651ab-31a5-452b-9583-c1d51f49dfcc', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7fab547695d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-28c651ab-31a5-452b-9583-c1d51f49dfcc', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:55:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-121709470',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=28c651ab-31a5-452b-9583-c1d51f49dfcc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:55:01Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:55:01Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'availability_zone': u'nova', u'reservations': [u'69e94f82-3fb2-452b-9811-f13aae70b3cf', u'e0c83de6-0416-4bb8-b982-8d350b290d90'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'065c0797e3cf47d08cced1bc93b84eb1', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestVolumeBootPattern-volume-origin-121709470'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'availability_zone': u'nova', u'reservations': [u'69e94f82-3fb2-452b-9811-f13aae70b3cf', u'e0c83de6-0416-4bb8-b982-8d350b290d90'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'065c0797e3cf47d08cced1bc93b84eb1', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestVolumeBootPattern-volume-origin-121709470'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'28c651ab-31a5-452b-9583-c1d51f49dfcc', u'size': 1, u'display_name': u'tempest-TestVolumeBootPattern-volume-origin-121709470', u'display_description': None, u'cluster_name': None, u'name_id': u'28c651ab-31a5-452b-9583-c1d51f49dfcc', u'volume_admin_metadata': [], u'project_id': u'065c0797e3cf47d08cced1bc93b84eb1', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-28c651ab-31a5-452b-9583-c1d51f49dfcc', u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'bootable': False, u'created_at': u'2018-08-29T05:55:01.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'28c651ab-31a5-452b-9583-c1d51f49dfcc', u'group_id': None}, u'user_id': u'aa9c107f505042bab157d4b3267c5185', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7fab54b7d7d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(28c651ab-31a5-452b-9583-c1d51f49dfcc),volume_id=28c651ab-31a5-452b-9583-c1d51f49dfcc,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:55:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-121709470',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=28c651ab-31a5-452b-9583-c1d51f49dfcc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:55:01Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:55:01Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fab54b7d7d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-29T05:55:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-121709470',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=28c651ab-31a5-452b-9583-c1d51f49dfcc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='065c0797e3cf47d08cced1bc93b84eb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-29T05:55:01Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-29T05:55:01Z,user_id='aa9c107f505042bab157d4b3267c5185',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager **volume_spec) >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager image_service) >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager return f(*a, **k) >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager image_service >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager image_location, image_service) >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:55:10.343 257256 ERROR cinder.volume.manager >2018-08-29 05:55:10.349 257256 WARNING cinder.volume.manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68a7a052-6865-433a-b377-4ddd2b87cfa9) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:55:10.352 257256 WARNING cinder.volume.manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cd7e2460-80a0-439e-8212-68abbed45fbf) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:55:10.355 257256 WARNING cinder.volume.manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7277a57b-e6da-4779-add5-24b1981a85d3) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:55:10.364 257256 ERROR cinder.volume.flows.manager.create_volume [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Volume 28c651ab-31a5-452b-9583-c1d51f49dfcc: create failed >2018-08-29 05:55:10.365 257256 WARNING cinder.volume.manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c70ba083-d138-4efe-acda-738adcabedad) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:55:10.368 257256 WARNING cinder.volume.manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca416a4a-0b74-4db4-885b-14c2044dcc8b) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-29 05:55:10.370 257256 WARNING cinder.volume.manager [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Flow 'volume_create_manager' (22679647-67f5-466e-8009-9cb767fb9e29) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server [req-1dcdd3a1-13ce-412d-945e-c3b97756a01f aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server image_service) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server image_service >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: We are unable to locate any Fibre Channel devices. >2018-08-29 05:55:10.376 257256 ERROR oslo_messaging.rpc.server >2018-08-29 05:55:12.285 257256 INFO cinder.volume.manager [req-122152fb-7632-47cc-90a7-b2858a7539d7 aa9c107f505042bab157d4b3267c5185 065c0797e3cf47d08cced1bc93b84eb1 - default default] Deleted volume successfully. >2018-08-29 05:55:26.887 257256 INFO cinder.volume.flows.manager.create_volume [req-f179fd6a-1dfa-4030-89a0-ebaac2780684 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Volume 25dbc37b-c65d-4189-8aff-fb870e3b836c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-25dbc37b-c65d-4189-8aff-fb870e3b836c'} >2018-08-29 05:55:27.423 257256 INFO cinder.volume.flows.manager.create_volume [req-f179fd6a-1dfa-4030-89a0-ebaac2780684 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Volume volume-25dbc37b-c65d-4189-8aff-fb870e3b836c (25dbc37b-c65d-4189-8aff-fb870e3b836c): created successfully >2018-08-29 05:55:27.426 257256 INFO cinder.volume.manager [req-f179fd6a-1dfa-4030-89a0-ebaac2780684 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Created volume successfully. >2018-08-29 05:55:35.559 257256 INFO cinder.volume.manager [req-40fd6252-edc8-4256-983f-ed9e0e7f4372 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Create snapshot completed successfully >2018-08-29 05:55:36.435 257256 INFO cinder.volume.manager [req-7b9f08ab-0dcc-4020-b095-e8c0c668f404 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Start to perform revert to snapshot process. >2018-08-29 05:55:48.643 257256 INFO cinder.volume.manager [req-7b9f08ab-0dcc-4020-b095-e8c0c668f404 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Create snapshot completed successfully >2018-08-29 05:55:48.748 257256 ERROR root [req-7b9f08ab-0dcc-4020-b095-e8c0c668f404 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Original exception being dropped: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 949, in revert_to_snapshot\n self._revert_to_snapshot(context, volume, snapshot)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 902, in _revert_to_snapshot\n self._notify_about_snapshot_usage(context, snapshot, "revert.start")\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2488, in _notify_about_snapshot_usage\n extra_usage_info=extra_usage_info, host=self.host)\n', ' File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 1105, in wrapped\n return f(*args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/utils.py", line 193, in notify_about_snapshot_usage\n usage_info = _usage_from_snapshot(snapshot, context, **extra_usage_info)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/utils.py", line 177, in _usage_from_snapshot\n \'metadata\': null_safe_str(snapshot.metadata),\n', ' File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 68, in getter\n return getattr(self, attrname)\n', "AttributeError: 'Snapshot' object has no attribute '_obj_metadata'\n"] >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server [req-7b9f08ab-0dcc-4020-b095-e8c0c668f404 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Exception during message handling: AttributeError: 'Snapshot' object has no attribute '_obj_metadata' >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 955, in revert_to_snapshot >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server "revert.end") >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2488, in _notify_about_snapshot_usage >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server extra_usage_info=extra_usage_info, host=self.host) >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 1105, in wrapped >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/utils.py", line 193, in notify_about_snapshot_usage >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server usage_info = _usage_from_snapshot(snapshot, context, **extra_usage_info) >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/utils.py", line 177, in _usage_from_snapshot >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server 'metadata': null_safe_str(snapshot.metadata), >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 68, in getter >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server return getattr(self, attrname) >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server AttributeError: 'Snapshot' object has no attribute '_obj_metadata' >2018-08-29 05:55:48.749 257256 ERROR oslo_messaging.rpc.server >2018-08-29 06:00:37.263 257256 INFO cinder.volume.flows.manager.create_volume [req-efc9dcc9-200b-44af-821f-16f3214aee05 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Volume 9fa47c3d-91b8-4f0b-866b-7bf9ece73b15: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9fa47c3d-91b8-4f0b-866b-7bf9ece73b15'} >2018-08-29 06:00:37.834 257256 INFO cinder.volume.flows.manager.create_volume [req-efc9dcc9-200b-44af-821f-16f3214aee05 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Volume volume-9fa47c3d-91b8-4f0b-866b-7bf9ece73b15 (9fa47c3d-91b8-4f0b-866b-7bf9ece73b15): created successfully >2018-08-29 06:00:37.838 257256 INFO cinder.volume.manager [req-efc9dcc9-200b-44af-821f-16f3214aee05 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Created volume successfully. >2018-08-29 06:00:45.882 257256 INFO cinder.volume.manager [req-70526c4e-a5bc-4509-97e4-7e77f73b563b 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Create snapshot completed successfully >2018-08-29 06:00:46.985 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-ce86aa84-925a-437b-af09-9f36be8c6bee 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] _extend_volume_op: Extending a volume with snapshots is not supported. >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager [req-ce86aa84-925a-437b-af09-9f36be8c6bee 7aa3b1692bf7417e8d2287be84c9b2ec d1e326de15b747c09158b52b0c3e5f70 - default default] Extend volume failed.: VolumeDriverException: Volume driver reported an error: _extend_volume_op: Extending a volume with snapshots is not supported. >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2543, in extend_volume >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size) >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2684, in extend_volume >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager self._extend_volume_op(volume, new_size) >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2695, in _extend_volume_op >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager raise exception.VolumeDriverException(message=msg) >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager VolumeDriverException: Volume driver reported an error: _extend_volume_op: Extending a volume with snapshots is not supported. >2018-08-29 06:00:46.986 257256 ERROR cinder.volume.manager >2018-08-29 06:04:13.342 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 06:04:13.344 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.345 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.346 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.346 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task command) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.346 257256 ERROR oslo_service.periodic_task >2018-08-29 06:04:13.407 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 06:04:13.408 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.408 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.409 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.409 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task command) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 06:04:13.410 257031 ERROR oslo_service.periodic_task >2018-08-29 06:04:43.356 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:04:43.421 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:04:43.457 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:04:43.509 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:05:43.355 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:05:43.424 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:05:43.428 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:05:43.494 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:06:43.354 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:06:43.430 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:06:43.434 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:06:43.501 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:07:43.358 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:07:43.445 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:07:43.449 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:07:43.532 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:34:13.338 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 06:34:13.340 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.341 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.341 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.341 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task command) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.342 257256 ERROR oslo_service.periodic_task >2018-08-29 06:34:13.436 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 06:34:13.437 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.438 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.438 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.438 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task command) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 06:34:13.439 257031 ERROR oslo_service.periodic_task >2018-08-29 06:34:43.344 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:34:43.420 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:34:43.447 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:34:43.532 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:35:43.335 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:35:43.398 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:35:43.460 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:35:43.543 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:36:43.339 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:36:43.437 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 06:36:43.449 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 06:36:43.508 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:04:13.328 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 07:04:13.331 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.331 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.331 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.332 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task command) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.332 257256 ERROR oslo_service.periodic_task >2018-08-29 07:04:13.435 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 07:04:13.437 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.437 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.437 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.438 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task command) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 07:04:13.438 257031 ERROR oslo_service.periodic_task >2018-08-29 07:04:43.342 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:04:43.431 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:04:43.453 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:04:43.536 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:05:43.341 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:05:43.425 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:05:43.449 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:05:43.518 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:36:43.353 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:36:43.473 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:36:44.368 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:36:44.369 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:37:43.363 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:37:43.438 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 07:37:43.468 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 07:37:43.546 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 08:04:13.325 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 08:04:13.327 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.328 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.328 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.328 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task command) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.329 257256 ERROR oslo_service.periodic_task >2018-08-29 08:04:13.467 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 08:04:13.468 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.468 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.469 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.469 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task command) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 08:04:13.469 257031 ERROR oslo_service.periodic_task >2018-08-29 08:04:43.329 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 08:04:43.395 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 08:04:43.477 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 08:04:43.552 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 08:34:13.335 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 08:34:13.337 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.338 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.338 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.338 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task command) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.339 257256 ERROR oslo_service.periodic_task >2018-08-29 08:34:13.465 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 08:34:13.466 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.467 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.467 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.467 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task command) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 08:34:13.468 257031 ERROR oslo_service.periodic_task >2018-08-29 08:34:43.356 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 08:34:43.441 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 08:34:43.500 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 08:34:43.606 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 09:04:13.322 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 09:04:13.325 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.326 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.326 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.326 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task command) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.327 257256 ERROR oslo_service.periodic_task >2018-08-29 09:04:13.449 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 09:04:13.450 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.450 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.451 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.451 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task command) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 09:04:13.451 257031 ERROR oslo_service.periodic_task >2018-08-29 09:04:43.329 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 09:04:43.434 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 09:04:43.457 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 09:04:43.520 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 09:34:13.348 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 09:34:13.351 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.351 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.352 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.352 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task command) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.352 257256 ERROR oslo_service.periodic_task >2018-08-29 09:34:13.476 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 09:34:13.477 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.477 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.478 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.478 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task command) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 09:34:13.478 257031 ERROR oslo_service.periodic_task >2018-08-29 09:34:43.502 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 09:34:43.626 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 09:35:43.372 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 09:35:43.448 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 10:04:13.352 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 10:04:13.354 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.354 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.355 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.355 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task command) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.355 257256 ERROR oslo_service.periodic_task >2018-08-29 10:04:13.449 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 10:04:13.450 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.450 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.451 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.451 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task command) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 10:04:13.451 257031 ERROR oslo_service.periodic_task >2018-08-29 10:04:43.356 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 10:04:43.445 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 10:04:43.460 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 10:04:43.550 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 10:34:13.397 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 10:34:13.400 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.400 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.401 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.401 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task command) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.401 257256 ERROR oslo_service.periodic_task >2018-08-29 10:34:13.461 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 10:34:13.462 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.462 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.462 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.463 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task command) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 10:34:13.463 257031 ERROR oslo_service.periodic_task >2018-08-29 10:34:43.403 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 10:34:43.491 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 10:34:43.992 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 10:34:44.047 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 11:04:13.388 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 11:04:13.390 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.391 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.391 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.391 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task command) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.392 257256 ERROR oslo_service.periodic_task >2018-08-29 11:04:13.479 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 11:04:13.480 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.480 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.480 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.481 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task command) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 11:04:13.481 257031 ERROR oslo_service.periodic_task >2018-08-29 11:04:43.407 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 11:04:43.489 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 11:04:43.879 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 11:04:43.937 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 11:34:13.407 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 11:34:13.410 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.410 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.410 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.411 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task command) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.411 257256 ERROR oslo_service.periodic_task >2018-08-29 11:34:13.478 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 11:34:13.479 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.480 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.480 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.480 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task command) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 11:34:13.481 257031 ERROR oslo_service.periodic_task >2018-08-29 11:34:43.409 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 11:34:43.478 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 11:34:44.920 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 11:34:45.236 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 12:04:13.398 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 12:04:13.400 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.400 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.401 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.401 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task command) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.401 257256 ERROR oslo_service.periodic_task >2018-08-29 12:04:13.473 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 12:04:13.474 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.475 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.475 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.475 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task command) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 12:04:13.476 257031 ERROR oslo_service.periodic_task >2018-08-29 12:04:43.425 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 12:04:43.502 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 12:04:44.182 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 12:04:44.234 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 12:34:13.394 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 12:34:13.396 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.396 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.397 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.397 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task command) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.397 257256 ERROR oslo_service.periodic_task >2018-08-29 12:34:13.496 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 12:34:13.497 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.497 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.497 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.498 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task command) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 12:34:13.498 257031 ERROR oslo_service.periodic_task >2018-08-29 12:34:43.405 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 12:34:43.503 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 12:34:44.434 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 12:34:44.751 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 13:04:13.411 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 13:04:13.413 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.414 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.414 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.414 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task command) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.415 257256 ERROR oslo_service.periodic_task >2018-08-29 13:04:13.536 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 13:04:13.537 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.538 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.538 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.538 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task command) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 13:04:13.539 257031 ERROR oslo_service.periodic_task >2018-08-29 13:04:43.413 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 13:04:43.549 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 13:04:44.847 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 13:04:45.166 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 13:34:13.410 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 13:34:13.413 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.413 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.413 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.414 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task command) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.414 257256 ERROR oslo_service.periodic_task >2018-08-29 13:34:13.526 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 13:34:13.527 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.527 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.528 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.528 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task command) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 13:34:13.528 257031 ERROR oslo_service.periodic_task >2018-08-29 13:34:43.422 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 13:34:43.534 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 13:34:43.645 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 13:34:43.937 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 14:04:13.430 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 14:04:13.433 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.433 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.433 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.434 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task command) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.434 257256 ERROR oslo_service.periodic_task >2018-08-29 14:04:13.533 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 14:04:13.533 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.534 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.534 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.534 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task command) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 14:04:13.535 257031 ERROR oslo_service.periodic_task >2018-08-29 14:04:43.444 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 14:04:43.542 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 14:04:45.471 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 14:04:45.781 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 14:34:13.439 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 14:34:13.442 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.442 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.443 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.443 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task command) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.443 257256 ERROR oslo_service.periodic_task >2018-08-29 14:34:13.550 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 14:34:13.551 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.552 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.552 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.552 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task command) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 14:34:13.553 257031 ERROR oslo_service.periodic_task >2018-08-29 14:34:43.454 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 14:34:43.562 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 14:34:45.216 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 14:34:45.510 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 15:04:13.443 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 15:04:13.445 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.446 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.446 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.447 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task command) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.447 257256 ERROR oslo_service.periodic_task >2018-08-29 15:04:13.532 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 15:04:13.533 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.533 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.534 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.534 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task command) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 15:04:13.534 257031 ERROR oslo_service.periodic_task >2018-08-29 15:04:43.456 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 15:04:43.540 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 15:04:44.396 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 15:04:44.687 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 15:34:13.460 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 15:34:13.462 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.463 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.463 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.463 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task command) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.464 257256 ERROR oslo_service.periodic_task >2018-08-29 15:34:13.549 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 15:34:13.550 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.550 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.551 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.551 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task command) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 15:34:13.551 257031 ERROR oslo_service.periodic_task >2018-08-29 15:34:43.474 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 15:34:43.559 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 15:34:43.883 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 15:34:43.973 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 16:04:13.483 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 16:04:13.486 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.486 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.486 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.487 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task command) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.487 257256 ERROR oslo_service.periodic_task >2018-08-29 16:04:13.535 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: timed out: timeout: timed out >2018-08-29 16:04:13.536 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.537 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.537 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.537 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task command) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: timed out >2018-08-29 16:04:13.537 257031 ERROR oslo_service.periodic_task >2018-08-29 16:04:43.489 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 16:04:43.545 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 16:04:43.551 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 16:04:43.609 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 16:33:43.525 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.526 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.526 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.527 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.527 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task command) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.527 257256 ERROR oslo_service.periodic_task >2018-08-29 16:33:43.566 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.567 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.567 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.568 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.568 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task command) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 16:33:43.568 257031 ERROR oslo_service.periodic_task >2018-08-29 16:34:43.543 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 16:34:43.579 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 16:34:43.611 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 16:34:43.652 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 17:03:43.554 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.556 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.556 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.556 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.557 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task command) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.557 257256 ERROR oslo_service.periodic_task >2018-08-29 17:03:43.584 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.585 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.585 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.585 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.586 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task command) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:03:43.586 257031 ERROR oslo_service.periodic_task >2018-08-29 17:04:43.574 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 17:04:43.598 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 17:04:43.635 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 17:04:43.658 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 17:33:43.543 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.544 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.545 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.545 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.545 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task command) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.546 257256 ERROR oslo_service.periodic_task >2018-08-29 17:33:43.589 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.589 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.589 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.590 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.590 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task command) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-29 17:33:43.590 257031 ERROR oslo_service.periodic_task >2018-08-29 17:34:43.555 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 17:34:43.609 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-29 17:34:43.676 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-29 17:34:43.712 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-30 01:22:43.737 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.738 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.739 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.739 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.739 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task command) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.740 257256 ERROR oslo_service.periodic_task >2018-08-30 01:22:43.788 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.789 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.789 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.790 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.790 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task command) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 01:22:43.790 257031 ERROR oslo_service.periodic_task >2018-08-30 01:23:43.753 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-30 01:23:43.801 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-30 01:23:43.829 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-30 01:23:43.869 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-30 02:12:58.809 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-30 02:12:58.879 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-30 02:12:58.890 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-30 02:12:58.954 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-30 04:54:43.818 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.820 257256 ERROR cinder.ssh_utils [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.820 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.820 257256 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.821 257256 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task [req-e3d1818a-28b9-47cf-b1f0-27a21d6e6186 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task task(self, context) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task command) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task obj = self.get() >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.821 257256 ERROR oslo_service.periodic_task >2018-08-30 04:54:43.867 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213: NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.868 257031 ERROR cinder.ssh_utils [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Create new item in SSHPool failed.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.868 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.868 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Not able to use storwize_san_secondary_ip since it is not configured.: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.869 257031 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error running SSH command: svcinfo lsmdiskgrp -bytes -delim ! "NonDR_DS8KBH00_Pool": SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task [req-0d4c04fa-e683-4776-9015-9cc441efe923 - - - - -] Error during VolumeManager._report_driver_status: SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task Traceback (most recent call last): >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task task(self, context) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task volume_stats = self.driver.get_volume_stats(refresh=True) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3781, in get_volume_stats >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task self._update_volume_stats() >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 3994, in _update_volume_stats >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task self._get_backend_pools()] >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 4006, in _build_pool_stats >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task pool_data = self._helpers.get_pool_attrs(pool) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 717, in get_pool_attrs >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task return self.ssh.lsmdiskgrp(pool) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 214, in lsmdiskgrp >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task return self.run_ssh_info(ssh_cmd)[0] >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 162, in run_ssh_info >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task raw = self._run_ssh(ssh_cmd) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 150, in _run_ssh >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task return self._ssh(ssh_cmd) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2405, in _run_ssh >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task command) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2382, in _run_ssh >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task check_exit_code, attempts) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2454, in _ssh_execute >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task LOG.error("Error running SSH command: %s", command) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2427, in _ssh_execute >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task with sshpool.item() as ssh: >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task return self.gen.next() >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/eventlet/pools.py", line 109, in item >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task obj = self.get() >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 176, in get >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task self.current_size -= 1 >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task self.force_reraise() >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task six.reraise(self.type_, self.value, self.tb) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 171, in get >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task new_conn = self.create() >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/cinder/ssh_utils.py", line 154, in create >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task raise paramiko.SSHException(msg) >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task SSHException: Error connecting via ssh: [Errno None] Unable to connect to port 22 on 9.11.110.213 >2018-08-30 04:54:43.869 257031 ERROR oslo_service.periodic_task >2018-08-30 04:55:43.849 257256 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-30 04:55:43.896 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-30 04:55:43.920 257256 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-30 04:55:43.960 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 01:57:29.745 257031 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-da5f4c63-6156-467f-8c1f-484a1a89ebc4 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] unmap_vol_from_host: No mapping of volume volume-27f45d41-1cc2-4a9f-8af7-49de9acada2e to any host found. >2018-08-31 01:57:29.747 257031 WARNING cinder.zonemanager.utils [req-da5f4c63-6156-467f-8c1f-484a1a89ebc4 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Driver didn't return connection info from terminate_connection call. >2018-08-31 01:57:29.747 257031 INFO cinder.volume.manager [req-da5f4c63-6156-467f-8c1f-484a1a89ebc4 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Terminate volume connection completed successfully. >2018-08-31 01:57:43.927 257031 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 01:57:43.984 257031 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 01:57:54.207 257031 WARNING cinder.quota [req-326b3ae5-939b-48a4-8edc-dd0819d7f116 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deprecated: Default quota for resource: gigabytes_svc1 is set by the default quota flag: quota_gigabytes_svc1, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 01:57:54.210 257031 WARNING cinder.quota [req-326b3ae5-939b-48a4-8edc-dd0819d7f116 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deprecated: Default quota for resource: volumes_svc1 is set by the default quota flag: quota_volumes_svc1, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 01:57:54.578 257031 INFO cinder.volume.manager [req-326b3ae5-939b-48a4-8edc-dd0819d7f116 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deleted volume successfully. >2018-08-31 01:59:06.018 256255 INFO oslo_service.service [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Caught SIGTERM, stopping children >2018-08-31 01:59:06.021 256255 INFO oslo_service.service [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Waiting on 2 children to exit >2018-08-31 01:59:43.942 256255 INFO oslo_service.service [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Child 257031 exited with status 0 >2018-08-31 01:59:44.042 256255 INFO oslo_service.service [req-6529da6f-e217-441e-8264-b3eb0954f6e4 - - - - -] Child 257256 exited with status 0 >2018-08-31 01:59:47.319 750660 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-31 01:59:47.984 750660 INFO cinder.rpc [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-31 01:59:47.989 750660 INFO cinder.rpc [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-31 01:59:48.002 750660 WARNING cinder.keymgr.conf_key_mgr [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-31 01:59:49.051 750660 INFO cinder.volume.manager [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-31 01:59:49.068 750660 INFO oslo_service.service [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Starting 1 workers >2018-08-31 01:59:49.080 751049 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-31 01:59:49.096 751049 INFO cinder.volume.manager [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Starting volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 01:59:49.124 751049 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 01:59:49.217 751049 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 01:59:50.099 750660 INFO cinder.volume.manager [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-31 01:59:50.115 750660 INFO oslo_service.service [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Starting 1 workers >2018-08-31 01:59:50.124 750660 WARNING oslo_config.cfg [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-31 01:59:50.125 751065 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-31 01:59:50.131 750660 WARNING oslo_config.cfg [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 01:59:50.132 750660 WARNING oslo_config.cfg [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 01:59:50.132 750660 WARNING oslo_config.cfg [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 01:59:50.141 751065 INFO cinder.volume.manager [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Starting volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 01:59:50.172 751065 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 01:59:50.278 751065 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 01:59:54.768 751049 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] WWPN on node 205: [u'500507680C384AFD', u'500507680C364AFD', u'500507680C114AFD', u'500507680C344AFD', u'500507680C324AFD', u'500507680C174AFD', u'500507680C334AFD', u'500507680C134AFD', u'500507680C154AFD', u'500507680C124AFD', u'500507680C354AFD', u'500507680C314AFD', u'500507680C184AFD', u'500507680C374AFD', u'500507680C144AFD', u'500507680C164AFD']. >2018-08-31 01:59:55.439 751049 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] WWPN on node 206: [u'500507680C1648C2', u'500507680C3648C2', u'500507680C3748C2', u'500507680C3848C2', u'500507680C1348C2', u'500507680C1848C2', u'500507680C3348C2', u'500507680C3548C2', u'500507680C1448C2', u'500507680C1548C2', u'500507680C1148C2', u'500507680C1748C2', u'500507680C1248C2', u'500507680C3248C2', u'500507680C3448C2', u'500507680C3148C2']. >2018-08-31 01:59:58.782 751065 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] WWPN on node 205: [u'500507680C384AFD', u'500507680C364AFD', u'500507680C114AFD', u'500507680C344AFD', u'500507680C324AFD', u'500507680C174AFD', u'500507680C334AFD', u'500507680C134AFD', u'500507680C154AFD', u'500507680C124AFD', u'500507680C354AFD', u'500507680C314AFD', u'500507680C184AFD', u'500507680C374AFD', u'500507680C144AFD', u'500507680C164AFD']. >2018-08-31 01:59:59.408 751065 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] WWPN on node 206: [u'500507680C1648C2', u'500507680C3648C2', u'500507680C3748C2', u'500507680C3848C2', u'500507680C1348C2', u'500507680C1848C2', u'500507680C3348C2', u'500507680C3548C2', u'500507680C1448C2', u'500507680C1548C2', u'500507680C1148C2', u'500507680C1748C2', u'500507680C1248C2', u'500507680C3248C2', u'500507680C3448C2', u'500507680C3148C2']. >2018-08-31 02:00:00.135 751049 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] ensure_export: Volume volume-e215809b-17c7-4f55-a533-faaa4177c25e not found on storage. >2018-08-31 02:00:00.884 751049 WARNING oslo_config.cfg [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:00.885 751049 WARNING oslo_config.cfg [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:00.885 751049 WARNING oslo_config.cfg [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:01.020 751049 INFO cinder.volume.manager [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Driver initialization completed successfully. >2018-08-31 02:00:01.028 751049 INFO cinder.manager [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Initiating service 6 cleanup >2018-08-31 02:00:01.032 751049 INFO cinder.manager [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Service 6 cleanup completed. >2018-08-31 02:00:01.311 751049 INFO cinder.volume.manager [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 02:00:01.395 751065 WARNING oslo_config.cfg [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:01.396 751065 WARNING oslo_config.cfg [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:01.396 751065 WARNING oslo_config.cfg [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 02:00:01.533 751065 INFO cinder.volume.manager [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Driver initialization completed successfully. >2018-08-31 02:00:01.539 751065 INFO cinder.manager [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Initiating service 7 cleanup >2018-08-31 02:00:01.543 751065 INFO cinder.manager [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Service 7 cleanup completed. >2018-08-31 02:00:01.820 751065 INFO cinder.volume.manager [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 02:00:01.940 751049 INFO cinder.volume.manager [req-6958213c-922c-4b5d-974b-683b06eaac8e - - - - -] Driver post RPC initialization completed successfully. >2018-08-31 02:00:02.374 751065 INFO cinder.volume.manager [req-c0eddab8-c19c-4b83-8d4e-00234e490ecc - - - - -] Driver post RPC initialization completed successfully. >2018-08-31 02:03:06.286 751065 INFO cinder.volume.flows.manager.create_volume [req-793a4067-c5aa-4219-980c-4255d23b2641 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume 7cb056d0-5f00-4718-bce6-6bc756a60c21: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7cb056d0-5f00-4718-bce6-6bc756a60c21'} >2018-08-31 02:03:06.934 751065 INFO cinder.volume.flows.manager.create_volume [req-793a4067-c5aa-4219-980c-4255d23b2641 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume volume-7cb056d0-5f00-4718-bce6-6bc756a60c21 (7cb056d0-5f00-4718-bce6-6bc756a60c21): created successfully >2018-08-31 02:03:06.938 751065 INFO cinder.volume.manager [req-793a4067-c5aa-4219-980c-4255d23b2641 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Created volume successfully. >2018-08-31 02:40:49.003 751065 INFO cinder.volume.manager [req-f9c7935a-cfde-4aa5-8221-f8bce7c11a90 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-126037358: creating >2018-08-31 02:40:49.032 751065 INFO cinder.volume.manager [req-f9c7935a-cfde-4aa5-8221-f8bce7c11a90 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-126037358: created successfully >2018-08-31 02:40:49.052 751065 INFO cinder.volume.manager [req-f9c7935a-cfde-4aa5-8221-f8bce7c11a90 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Create group completed successfully. >2018-08-31 02:40:50.191 751065 INFO cinder.volume.flows.manager.create_volume [req-b8546ba1-fc1d-462a-a609-4c077c337642 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume 50d3183c-e60d-42a5-8d8e-d19a949cd214: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-50d3183c-e60d-42a5-8d8e-d19a949cd214'} >2018-08-31 02:40:50.817 751065 INFO cinder.volume.flows.manager.create_volume [req-b8546ba1-fc1d-462a-a609-4c077c337642 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume volume-50d3183c-e60d-42a5-8d8e-d19a949cd214 (50d3183c-e60d-42a5-8d8e-d19a949cd214): created successfully >2018-08-31 02:40:50.821 751065 INFO cinder.volume.manager [req-b8546ba1-fc1d-462a-a609-4c077c337642 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Created volume successfully. >2018-08-31 02:40:54.133 751065 INFO cinder.volume.manager [req-258fb9d7-6f85-43e9-8eea-97f2c6d323c4 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] GroupSnapshot 79a40640-3862-4663-87dd-3902290cd9fa: creating. >2018-08-31 02:41:01.473 751065 INFO cinder.volume.manager [req-258fb9d7-6f85-43e9-8eea-97f2c6d323c4 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] group_snapshot 79a40640-3862-4663-87dd-3902290cd9fa: created successfully >2018-08-31 02:41:01.724 751065 INFO cinder.volume.manager [req-fc1096fd-ef21-4b88-9fc4-ca5cc419c434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] group_snapshot 79a40640-3862-4663-87dd-3902290cd9fa: deleting >2018-08-31 02:41:13.688 751065 WARNING cinder.quota [req-fc1096fd-ef21-4b88-9fc4-ca5cc419c434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-1637608737 is set by the default quota flag: quota_snapshots_tempest-volume-type-1637608737, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:13.723 751065 INFO cinder.volume.manager [req-fc1096fd-ef21-4b88-9fc4-ca5cc419c434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] group_snapshot 79a40640-3862-4663-87dd-3902290cd9fa: deleted successfully >2018-08-31 02:41:16.273 751065 WARNING cinder.quota [req-986a1f5f-b9e7-4f82-b67d-fda94cfe83b6 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-1637608737 is set by the default quota flag: quota_volumes_tempest-volume-type-1637608737, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:16.274 751065 WARNING cinder.quota [req-986a1f5f-b9e7-4f82-b67d-fda94cfe83b6 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-1637608737 is set by the default quota flag: quota_gigabytes_tempest-volume-type-1637608737, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:16.624 751065 INFO cinder.volume.manager [req-986a1f5f-b9e7-4f82-b67d-fda94cfe83b6 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Delete group completed successfully. >2018-08-31 02:41:16.678 751065 INFO cinder.volume.manager [req-0d72b470-d426-48de-9295-7314fac77434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-1355516726: creating >2018-08-31 02:41:16.705 751065 INFO cinder.volume.manager [req-0d72b470-d426-48de-9295-7314fac77434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-1355516726: created successfully >2018-08-31 02:41:16.731 751065 INFO cinder.volume.manager [req-0d72b470-d426-48de-9295-7314fac77434 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Create group completed successfully. >2018-08-31 02:41:17.142 751065 INFO cinder.volume.flows.manager.create_volume [req-377d84a9-65aa-4001-b315-405445c1c18f 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume 19590387-6abf-4649-9f21-ddb524e110c9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-19590387-6abf-4649-9f21-ddb524e110c9'} >2018-08-31 02:41:17.838 751065 INFO cinder.volume.flows.manager.create_volume [req-377d84a9-65aa-4001-b315-405445c1c18f 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume volume-19590387-6abf-4649-9f21-ddb524e110c9 (19590387-6abf-4649-9f21-ddb524e110c9): created successfully >2018-08-31 02:41:17.842 751065 INFO cinder.volume.manager [req-377d84a9-65aa-4001-b315-405445c1c18f 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Created volume successfully. >2018-08-31 02:41:25.128 751065 WARNING cinder.quota [req-18fdacf7-6b1f-48bb-93fa-fe7dd17b9ac9 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-136632385 is set by the default quota flag: quota_gigabytes_tempest-volume-type-136632385, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:25.129 751065 WARNING cinder.quota [req-18fdacf7-6b1f-48bb-93fa-fe7dd17b9ac9 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-136632385 is set by the default quota flag: quota_volumes_tempest-volume-type-136632385, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:25.660 751065 INFO cinder.volume.manager [req-18fdacf7-6b1f-48bb-93fa-fe7dd17b9ac9 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Delete group completed successfully. >2018-08-31 02:41:25.670 751049 INFO cinder.volume.manager [req-deb51cf6-53f3-4809-ad72-a6c5d440bdaa 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-642265519: creating >2018-08-31 02:41:25.698 751049 INFO cinder.volume.manager [req-deb51cf6-53f3-4809-ad72-a6c5d440bdaa 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-642265519: created successfully >2018-08-31 02:41:25.747 751049 INFO cinder.volume.manager [req-deb51cf6-53f3-4809-ad72-a6c5d440bdaa 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Create group completed successfully. >2018-08-31 02:41:26.150 751049 INFO cinder.volume.flows.manager.create_volume [req-6c351b09-ab67-461c-bfb3-8c029c2021a1 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume cdd88319-efdf-4301-85bb-d0969740bdf7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cdd88319-efdf-4301-85bb-d0969740bdf7'} >2018-08-31 02:41:26.757 751049 INFO cinder.volume.flows.manager.create_volume [req-6c351b09-ab67-461c-bfb3-8c029c2021a1 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume volume-cdd88319-efdf-4301-85bb-d0969740bdf7 (cdd88319-efdf-4301-85bb-d0969740bdf7): created successfully >2018-08-31 02:41:26.760 751049 INFO cinder.volume.manager [req-6c351b09-ab67-461c-bfb3-8c029c2021a1 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Created volume successfully. >2018-08-31 02:41:27.280 751049 INFO cinder.volume.manager [req-ce242fe3-328f-4b44-a2f5-2c35cd89e359 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] GroupSnapshot ef76d328-5731-47d7-9d45-6f4d3f90a3bc: creating. >2018-08-31 02:41:34.643 751049 INFO cinder.volume.manager [req-ce242fe3-328f-4b44-a2f5-2c35cd89e359 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] group_snapshot ef76d328-5731-47d7-9d45-6f4d3f90a3bc: created successfully >2018-08-31 02:41:43.973 751049 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 02:41:44.031 751049 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 02:41:44.994 751049 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-76c8eaea-cc03-4a47-8ab0-6412fbc1b2bd 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Updating status for CG: efd5a274-8ad7-471c-9534-96aa95bbd39b. >2018-08-31 02:41:45.097 751049 INFO cinder.volume.manager [req-76c8eaea-cc03-4a47-8ab0-6412fbc1b2bd 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Create group from source-snapshot-ef76d328-5731-47d7-9d45-6f4d3f90a3bc completed successfully. >2018-08-31 02:41:49.555 751049 WARNING cinder.quota [req-08cf8f6f-fcd9-4e5f-a345-2567ec2773f5 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-159793540 is set by the default quota flag: quota_gigabytes_tempest-volume-type-159793540, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:49.556 751049 WARNING cinder.quota [req-08cf8f6f-fcd9-4e5f-a345-2567ec2773f5 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-159793540 is set by the default quota flag: quota_volumes_tempest-volume-type-159793540, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:49.864 751049 INFO cinder.volume.manager [req-08cf8f6f-fcd9-4e5f-a345-2567ec2773f5 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Delete group completed successfully. >2018-08-31 02:41:50.175 751049 INFO cinder.volume.manager [req-10470943-ec44-4503-b2ba-855302973da5 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] group_snapshot ef76d328-5731-47d7-9d45-6f4d3f90a3bc: deleting >2018-08-31 02:41:57.558 751049 WARNING cinder.quota [req-10470943-ec44-4503-b2ba-855302973da5 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: snapshots_tempest-volume-type-159793540 is set by the default quota flag: quota_snapshots_tempest-volume-type-159793540, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:41:57.592 751049 INFO cinder.volume.manager [req-10470943-ec44-4503-b2ba-855302973da5 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] group_snapshot ef76d328-5731-47d7-9d45-6f4d3f90a3bc: deleted successfully >2018-08-31 02:41:59.378 751049 INFO cinder.volume.manager [req-2810f190-8013-4d34-92ff-5a2f8720caf4 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Delete group completed successfully. >2018-08-31 02:41:59.592 751065 INFO cinder.volume.manager [req-c719ccd4-58cd-4089-aab7-a571eb21ba75 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-57183801: creating >2018-08-31 02:41:59.614 751065 INFO cinder.volume.manager [req-c719ccd4-58cd-4089-aab7-a571eb21ba75 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Group tempest-CG-57183801: created successfully >2018-08-31 02:41:59.634 751065 INFO cinder.volume.manager [req-c719ccd4-58cd-4089-aab7-a571eb21ba75 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Create group completed successfully. >2018-08-31 02:42:00.026 751065 INFO cinder.volume.flows.manager.create_volume [req-8700690f-606b-4ea4-928a-c18ee6faa3f3 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume 009f9043-b940-4175-ba4c-df37c7ddae72: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-009f9043-b940-4175-ba4c-df37c7ddae72'} >2018-08-31 02:42:00.629 751065 INFO cinder.volume.flows.manager.create_volume [req-8700690f-606b-4ea4-928a-c18ee6faa3f3 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Volume volume-009f9043-b940-4175-ba4c-df37c7ddae72 (009f9043-b940-4175-ba4c-df37c7ddae72): created successfully >2018-08-31 02:42:00.633 751065 INFO cinder.volume.manager [req-8700690f-606b-4ea4-928a-c18ee6faa3f3 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Created volume successfully. >2018-08-31 02:42:06.765 751065 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-972230b3-7303-4402-9512-1db713e488fb 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Updating status for CG: b15a8f04-cacf-4d60-ba8a-c8a5c829e276. >2018-08-31 02:42:06.861 751065 INFO cinder.volume.manager [req-972230b3-7303-4402-9512-1db713e488fb 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Create group from source-group-d808d47b-3ec1-4f53-a3af-953c1ab2fe0e completed successfully. >2018-08-31 02:42:19.472 751065 WARNING cinder.quota [req-f49aa406-8df5-449a-959a-25eb6cd83ff8 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: volumes_tempest-volume-type-288051993 is set by the default quota flag: quota_volumes_tempest-volume-type-288051993, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:42:19.473 751065 WARNING cinder.quota [req-f49aa406-8df5-449a-959a-25eb6cd83ff8 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Deprecated: Default quota for resource: gigabytes_tempest-volume-type-288051993 is set by the default quota flag: quota_gigabytes_tempest-volume-type-288051993, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:42:19.743 751065 INFO cinder.volume.manager [req-f49aa406-8df5-449a-959a-25eb6cd83ff8 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Delete group completed successfully. >2018-08-31 02:42:21.368 751065 INFO cinder.volume.manager [req-ba312224-eeb0-47ae-b056-1f43c3c89a24 1515aa02f9dc443ebef480fbf6006e49 e1b8f218465b40cda896b782c5d3cafe - default default] Delete group completed successfully. >2018-08-31 02:45:07.940 751065 INFO cinder.volume.flows.manager.create_volume [req-c7332bc2-8ee5-4b0f-a3f8-59d2817f1860 3054623f2a4c4d8a99ee1ada1e41e911 7f2ae49bf0df4e4cbe931f47423d2039 - default default] Volume 4c78f624-8c5a-431e-99cc-2388d18c2e55: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4c78f624-8c5a-431e-99cc-2388d18c2e55'} >2018-08-31 02:45:08.531 751065 INFO cinder.volume.flows.manager.create_volume [req-c7332bc2-8ee5-4b0f-a3f8-59d2817f1860 3054623f2a4c4d8a99ee1ada1e41e911 7f2ae49bf0df4e4cbe931f47423d2039 - default default] Volume volume-4c78f624-8c5a-431e-99cc-2388d18c2e55 (4c78f624-8c5a-431e-99cc-2388d18c2e55): created successfully >2018-08-31 02:45:08.534 751065 INFO cinder.volume.manager [req-c7332bc2-8ee5-4b0f-a3f8-59d2817f1860 3054623f2a4c4d8a99ee1ada1e41e911 7f2ae49bf0df4e4cbe931f47423d2039 - default default] Created volume successfully. >2018-08-31 02:45:12.523 751065 INFO cinder.volume.manager [req-a252a48d-aca4-41e6-8c78-33b2617fd3ff 3054623f2a4c4d8a99ee1ada1e41e911 7f2ae49bf0df4e4cbe931f47423d2039 - default default] Deleted volume successfully. >2018-08-31 02:45:21.889 751065 INFO cinder.volume.flows.manager.create_volume [req-d8fe4671-3b37-4265-8f10-bbbaa19eac2d 89923d3b985641dcabe3a7fc12f2ec5a b8bf2ec5e1be4ca4a1a77eaac00d79a3 - default default] Volume cd7f37c8-4bae-4313-b4b0-52a9497f06c2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cd7f37c8-4bae-4313-b4b0-52a9497f06c2'} >2018-08-31 02:45:22.526 751065 INFO cinder.volume.flows.manager.create_volume [req-d8fe4671-3b37-4265-8f10-bbbaa19eac2d 89923d3b985641dcabe3a7fc12f2ec5a b8bf2ec5e1be4ca4a1a77eaac00d79a3 - default default] Volume volume-cd7f37c8-4bae-4313-b4b0-52a9497f06c2 (cd7f37c8-4bae-4313-b4b0-52a9497f06c2): created successfully >2018-08-31 02:45:22.530 751065 INFO cinder.volume.manager [req-d8fe4671-3b37-4265-8f10-bbbaa19eac2d 89923d3b985641dcabe3a7fc12f2ec5a b8bf2ec5e1be4ca4a1a77eaac00d79a3 - default default] Created volume successfully. >2018-08-31 02:45:28.498 751065 WARNING cinder.quota [req-ac34f4cc-d5a5-481b-84f5-92b3a24dd10b 89923d3b985641dcabe3a7fc12f2ec5a b8bf2ec5e1be4ca4a1a77eaac00d79a3 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1226325445 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1226325445, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:45:28.499 751065 WARNING cinder.quota [req-ac34f4cc-d5a5-481b-84f5-92b3a24dd10b 89923d3b985641dcabe3a7fc12f2ec5a b8bf2ec5e1be4ca4a1a77eaac00d79a3 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1226325445 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1226325445, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 02:45:28.841 751065 INFO cinder.volume.manager [req-ac34f4cc-d5a5-481b-84f5-92b3a24dd10b 89923d3b985641dcabe3a7fc12f2ec5a b8bf2ec5e1be4ca4a1a77eaac00d79a3 - default default] Deleted volume successfully. >2018-08-31 02:46:24.374 751065 INFO cinder.volume.flows.manager.create_volume [req-81053a55-2096-445a-b325-f4549f350ea4 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume 428b6cf9-1276-4954-b8d9-caaba1db95c7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-428b6cf9-1276-4954-b8d9-caaba1db95c7'} >2018-08-31 02:46:24.980 751065 INFO cinder.volume.flows.manager.create_volume [req-81053a55-2096-445a-b325-f4549f350ea4 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume volume-428b6cf9-1276-4954-b8d9-caaba1db95c7 (428b6cf9-1276-4954-b8d9-caaba1db95c7): created successfully >2018-08-31 02:46:24.984 751065 INFO cinder.volume.manager [req-81053a55-2096-445a-b325-f4549f350ea4 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Created volume successfully. >2018-08-31 02:46:26.230 751065 INFO cinder.volume.manager [req-b50fc36c-e15c-4fb8-ae87-105222a200a3 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Attaching volume 428b6cf9-1276-4954-b8d9-caaba1db95c7 to instance 493c4cbc-0499-4d64-9101-6e3bfca20918 at mountpoint /dev/vdb on host None. >2018-08-31 02:46:26.318 751065 INFO cinder.volume.manager [req-b50fc36c-e15c-4fb8-ae87-105222a200a3 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Attach volume completed successfully. >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager [req-9220eb6c-18db-4d06-96d7-3e9aeef0eff4 ccb8693c6afc4669b0efa1291540923a b3b88971866b4107a47ea532b53fd4c6 - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager force=force) >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_iscsi.py", line 311, in terminate_connection >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-31 02:46:27.884 751065 ERROR cinder.volume.manager >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server [req-9220eb6c-18db-4d06-96d7-3e9aeef0eff4 ccb8693c6afc4669b0efa1291540923a b3b88971866b4107a47ea532b53fd4c6 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-31 02:46:27.904 751065 ERROR oslo_messaging.rpc.server >2018-08-31 02:46:28.188 751065 INFO cinder.volume.manager [req-39a834a4-c6f4-4a9e-83e0-c086d8e988ea 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Detaching volume 428b6cf9-1276-4954-b8d9-caaba1db95c7 from instance 493c4cbc-0499-4d64-9101-6e3bfca20918. >2018-08-31 02:46:28.287 751065 INFO cinder.volume.manager [req-39a834a4-c6f4-4a9e-83e0-c086d8e988ea 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Detach volume completed successfully. >2018-08-31 02:46:31.600 751049 INFO cinder.volume.flows.manager.create_volume [req-dfb34f34-898d-41df-b0fd-71344e1a923b 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume 654ba178-b096-472a-9a19-06bd0ef153b6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-654ba178-b096-472a-9a19-06bd0ef153b6'} >2018-08-31 02:46:32.353 751049 INFO cinder.volume.flows.manager.create_volume [req-dfb34f34-898d-41df-b0fd-71344e1a923b 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume volume-654ba178-b096-472a-9a19-06bd0ef153b6 (654ba178-b096-472a-9a19-06bd0ef153b6): created successfully >2018-08-31 02:46:32.357 751049 INFO cinder.volume.manager [req-dfb34f34-898d-41df-b0fd-71344e1a923b 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Created volume successfully. >2018-08-31 02:46:35.332 751049 INFO cinder.volume.manager [req-d79710e2-05e9-4108-92b9-d225c0240705 ccb8693c6afc4669b0efa1291540923a b3b88971866b4107a47ea532b53fd4c6 - default default] Deleted volume successfully. >2018-08-31 02:46:36.372 751065 INFO cinder.volume.flows.manager.create_volume [req-c2703ff9-f70a-4c2a-994d-ddd85e7fe645 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume 4ea3c0fd-1c54-48ea-bae3-97b2e6798e26: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4ea3c0fd-1c54-48ea-bae3-97b2e6798e26'} >2018-08-31 02:46:37.170 751065 INFO cinder.volume.flows.manager.create_volume [req-c2703ff9-f70a-4c2a-994d-ddd85e7fe645 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume volume-4ea3c0fd-1c54-48ea-bae3-97b2e6798e26 (4ea3c0fd-1c54-48ea-bae3-97b2e6798e26): created successfully >2018-08-31 02:46:37.174 751065 INFO cinder.volume.manager [req-c2703ff9-f70a-4c2a-994d-ddd85e7fe645 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Created volume successfully. >2018-08-31 02:46:38.410 751065 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 02:46:38.499 751065 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 02:46:42.424 751065 INFO cinder.volume.manager [req-b3a4d518-e498-49a4-a78c-8fd095bbe489 ccb8693c6afc4669b0efa1291540923a b3b88971866b4107a47ea532b53fd4c6 - default default] Deleted volume successfully. >2018-08-31 02:46:42.783 751065 INFO cinder.volume.flows.manager.create_volume [req-e4cdc14e-68b7-4e93-a617-8a363bf3dd67 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume 424c367e-181f-4b70-8222-e22d4f2251fd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-424c367e-181f-4b70-8222-e22d4f2251fd'} >2018-08-31 02:46:43.593 751065 INFO cinder.volume.flows.manager.create_volume [req-e4cdc14e-68b7-4e93-a617-8a363bf3dd67 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume volume-424c367e-181f-4b70-8222-e22d4f2251fd (424c367e-181f-4b70-8222-e22d4f2251fd): created successfully >2018-08-31 02:46:43.597 751065 INFO cinder.volume.manager [req-e4cdc14e-68b7-4e93-a617-8a363bf3dd67 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Created volume successfully. >2018-08-31 02:46:49.904 751065 INFO cinder.volume.manager [req-3b73c180-960b-4404-9c2c-60cc086c5a10 ccb8693c6afc4669b0efa1291540923a b3b88971866b4107a47ea532b53fd4c6 - default default] Deleted volume successfully. >2018-08-31 02:46:50.039 751049 INFO cinder.volume.flows.manager.create_volume [req-ca9d34cb-95a7-4fb6-90c7-79ff5b49c2ea 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume 70f99d78-3a56-4fd0-9fff-df9d328251e5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-70f99d78-3a56-4fd0-9fff-df9d328251e5'} >2018-08-31 02:46:51.078 751049 INFO cinder.volume.flows.manager.create_volume [req-ca9d34cb-95a7-4fb6-90c7-79ff5b49c2ea 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume volume-70f99d78-3a56-4fd0-9fff-df9d328251e5 (70f99d78-3a56-4fd0-9fff-df9d328251e5): created successfully >2018-08-31 02:46:51.081 751049 INFO cinder.volume.manager [req-ca9d34cb-95a7-4fb6-90c7-79ff5b49c2ea 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Created volume successfully. >2018-08-31 02:46:58.119 751049 INFO cinder.volume.manager [req-de771059-3b67-4b29-b3c7-f64a3a45a0da ccb8693c6afc4669b0efa1291540923a b3b88971866b4107a47ea532b53fd4c6 - default default] Deleted volume successfully. >2018-08-31 02:46:58.321 751065 INFO cinder.volume.flows.manager.create_volume [req-fb83e525-1373-4b59-bfc9-af8287ec8b54 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume f25c5bee-4a5e-4718-a6c6-ab568c48bd21: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f25c5bee-4a5e-4718-a6c6-ab568c48bd21'} >2018-08-31 02:46:59.277 751065 INFO cinder.volume.flows.manager.create_volume [req-fb83e525-1373-4b59-bfc9-af8287ec8b54 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Volume volume-f25c5bee-4a5e-4718-a6c6-ab568c48bd21 (f25c5bee-4a5e-4718-a6c6-ab568c48bd21): created successfully >2018-08-31 02:46:59.282 751065 INFO cinder.volume.manager [req-fb83e525-1373-4b59-bfc9-af8287ec8b54 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Created volume successfully. >2018-08-31 02:47:04.154 751065 INFO cinder.volume.manager [req-db965453-be05-4dea-8c70-51ba380bba7b 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Deleted volume successfully. >2018-08-31 02:47:04.491 751065 INFO cinder.volume.manager [req-b6b0e168-eba1-4ad3-b77a-1db75e7f45bc 79fc2ce8c7084f84a86eac8be726441f 47db68b6e6ed4209bbb05c6091442c23 - default default] Deleted volume successfully. >2018-08-31 02:47:11.889 751065 INFO cinder.volume.flows.manager.create_volume [req-b86f6d4c-4dde-4a09-9ed2-a65d76833af8 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Volume 627fa6ae-8db4-402d-8d57-9dd16e2a8fd1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-627fa6ae-8db4-402d-8d57-9dd16e2a8fd1'} >2018-08-31 02:47:12.883 751065 INFO cinder.volume.flows.manager.create_volume [req-b86f6d4c-4dde-4a09-9ed2-a65d76833af8 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Volume volume-627fa6ae-8db4-402d-8d57-9dd16e2a8fd1 (627fa6ae-8db4-402d-8d57-9dd16e2a8fd1): created successfully >2018-08-31 02:47:12.886 751065 INFO cinder.volume.manager [req-b86f6d4c-4dde-4a09-9ed2-a65d76833af8 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Created volume successfully. >2018-08-31 02:47:13.944 751065 INFO cinder.volume.flows.manager.create_volume [req-5604498d-944b-4255-b60c-fe27629ebe1c 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Volume 38d230ae-f15a-4cfc-a3b5-aa4957ad4aa1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-38d230ae-f15a-4cfc-a3b5-aa4957ad4aa1'} >2018-08-31 02:47:14.837 751065 INFO cinder.volume.flows.manager.create_volume [req-5604498d-944b-4255-b60c-fe27629ebe1c 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Volume volume-38d230ae-f15a-4cfc-a3b5-aa4957ad4aa1 (38d230ae-f15a-4cfc-a3b5-aa4957ad4aa1): created successfully >2018-08-31 02:47:14.841 751065 INFO cinder.volume.manager [req-5604498d-944b-4255-b60c-fe27629ebe1c 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Created volume successfully. >2018-08-31 02:47:15.812 751049 INFO cinder.volume.flows.manager.create_volume [req-80c9af07-2d18-4fea-a761-9e22ca422009 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Volume 7b985081-0300-44c7-828c-3d169eb44c99: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7b985081-0300-44c7-828c-3d169eb44c99'} >2018-08-31 02:47:20.059 751049 INFO cinder.volume.flows.manager.create_volume [req-80c9af07-2d18-4fea-a761-9e22ca422009 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Volume volume-7b985081-0300-44c7-828c-3d169eb44c99 (7b985081-0300-44c7-828c-3d169eb44c99): created successfully >2018-08-31 02:47:20.064 751049 INFO cinder.volume.manager [req-80c9af07-2d18-4fea-a761-9e22ca422009 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Created volume successfully. >2018-08-31 02:47:21.151 751065 INFO cinder.volume.flows.manager.create_volume [req-178ba50e-bb37-4198-8482-e03ec35aa4d2 ce248749952a42d3bcc0e30c0f7dd523 91e93bd56e4c465b85380731e8f5f2a4 - default default] Volume 6f2ddf3e-15f8-4e8d-8f03-891f745cb209: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6f2ddf3e-15f8-4e8d-8f03-891f745cb209'} >2018-08-31 02:47:22.090 751065 INFO cinder.volume.flows.manager.create_volume [req-178ba50e-bb37-4198-8482-e03ec35aa4d2 ce248749952a42d3bcc0e30c0f7dd523 91e93bd56e4c465b85380731e8f5f2a4 - default default] Volume volume-6f2ddf3e-15f8-4e8d-8f03-891f745cb209 (6f2ddf3e-15f8-4e8d-8f03-891f745cb209): created successfully >2018-08-31 02:47:22.094 751065 INFO cinder.volume.manager [req-178ba50e-bb37-4198-8482-e03ec35aa4d2 ce248749952a42d3bcc0e30c0f7dd523 91e93bd56e4c465b85380731e8f5f2a4 - default default] Created volume successfully. >2018-08-31 02:47:23.409 751065 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 02:47:23.491 751065 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 02:47:29.557 751065 INFO cinder.volume.manager [req-c86616c8-123e-4ee0-a714-4efce4600f5d ce248749952a42d3bcc0e30c0f7dd523 91e93bd56e4c465b85380731e8f5f2a4 - default default] Deleted volume successfully. >2018-08-31 02:47:29.575 751065 INFO cinder.volume.manager [req-d01abb5c-c5f0-43dd-8668-76b4c49c4202 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Deleted volume successfully. >2018-08-31 02:47:29.827 751049 INFO cinder.volume.manager [req-e5bb24a9-8b74-43e5-b893-3755d4b7b0ad 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Deleted volume successfully. >2018-08-31 02:47:30.100 751065 INFO cinder.volume.manager [req-b6e39909-770f-4ca3-bb67-eb162a306221 50288e0a0b064852a29ef32224449112 71ed32cf2bfb412e9bd9948e2507c5c0 - default default] Deleted volume successfully. >2018-08-31 02:47:35.869 751065 INFO cinder.volume.flows.manager.create_volume [req-f5c3c720-8408-48d4-a8b4-85252d17c755 17a37e768d384cd5b1f5646da76c9ce8 d7ec77f30ea64f9bbc8da92e8fc9428b - default default] Volume 32f87c52-74e5-4228-a3e3-c0fa15b518c8: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-32f87c52-74e5-4228-a3e3-c0fa15b518c8'} >2018-08-31 02:47:36.857 751065 INFO cinder.volume.flows.manager.create_volume [req-f5c3c720-8408-48d4-a8b4-85252d17c755 17a37e768d384cd5b1f5646da76c9ce8 d7ec77f30ea64f9bbc8da92e8fc9428b - default default] Volume volume-32f87c52-74e5-4228-a3e3-c0fa15b518c8 (32f87c52-74e5-4228-a3e3-c0fa15b518c8): created successfully >2018-08-31 02:47:36.862 751065 INFO cinder.volume.manager [req-f5c3c720-8408-48d4-a8b4-85252d17c755 17a37e768d384cd5b1f5646da76c9ce8 d7ec77f30ea64f9bbc8da92e8fc9428b - default default] Created volume successfully. >2018-08-31 02:47:45.677 751065 INFO cinder.volume.manager [req-0b4102c8-e046-452d-b02f-f3597acabc67 17a37e768d384cd5b1f5646da76c9ce8 d7ec77f30ea64f9bbc8da92e8fc9428b - default default] Deleted volume successfully. >2018-08-31 02:47:52.745 751065 INFO cinder.volume.flows.manager.create_volume [req-ad679d31-5cd5-43b0-9a7f-7abbddc182f8 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] Volume 272b80a6-982c-44b4-a07d-0162b0dd7f49: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-272b80a6-982c-44b4-a07d-0162b0dd7f49'} >2018-08-31 02:47:53.729 751065 INFO cinder.volume.flows.manager.create_volume [req-ad679d31-5cd5-43b0-9a7f-7abbddc182f8 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] Volume volume-272b80a6-982c-44b4-a07d-0162b0dd7f49 (272b80a6-982c-44b4-a07d-0162b0dd7f49): created successfully >2018-08-31 02:47:53.732 751065 INFO cinder.volume.manager [req-ad679d31-5cd5-43b0-9a7f-7abbddc182f8 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] Created volume successfully. >2018-08-31 02:47:55.600 751065 INFO cinder.volume.manager [req-2c818010-9863-4d20-b76d-10ddb32a85d1 c033bef8d1604331ada7fa7536978dec 365e86f41ff44621855b02b64f0b6ffc - default default] Transfer volume completed successfully. >2018-08-31 02:47:58.620 751065 INFO cinder.volume.manager [req-f84995ba-96c4-4d59-9df5-c9e48dca5197 58d124b1aa1b46e4ac1f39816be14504 0cfc5a33c89b4662b10af051f65dedec - default default] Deleted volume successfully. >2018-08-31 02:47:59.857 751065 INFO cinder.volume.flows.manager.create_volume [req-853491c2-a4bf-436d-820d-8cbe36936b5f 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] Volume f22282f0-4cfc-43ef-9118-138c3b7ca19a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f22282f0-4cfc-43ef-9118-138c3b7ca19a'} >2018-08-31 02:48:00.749 751065 INFO cinder.volume.flows.manager.create_volume [req-853491c2-a4bf-436d-820d-8cbe36936b5f 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] Volume volume-f22282f0-4cfc-43ef-9118-138c3b7ca19a (f22282f0-4cfc-43ef-9118-138c3b7ca19a): created successfully >2018-08-31 02:48:00.753 751065 INFO cinder.volume.manager [req-853491c2-a4bf-436d-820d-8cbe36936b5f 04dca9da40a54fa19d788fdec60212aa 9350b6efa0b543fca897df0a48bde4d2 - default default] Created volume successfully. >2018-08-31 02:48:03.575 751065 INFO cinder.volume.manager [req-ac691473-aab5-42bf-864b-9513393dc6b2 58d124b1aa1b46e4ac1f39816be14504 0cfc5a33c89b4662b10af051f65dedec - default default] Deleted volume successfully. >2018-08-31 02:48:10.216 751065 INFO cinder.volume.flows.manager.create_volume [req-5daf5729-9dc5-42e7-88c1-00a98cea8564 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Volume 4c5385c7-2189-41f2-b549-16ec5b9240f0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4c5385c7-2189-41f2-b549-16ec5b9240f0'} >2018-08-31 02:48:11.140 751065 INFO cinder.volume.flows.manager.create_volume [req-5daf5729-9dc5-42e7-88c1-00a98cea8564 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Volume volume-4c5385c7-2189-41f2-b549-16ec5b9240f0 (4c5385c7-2189-41f2-b549-16ec5b9240f0): created successfully >2018-08-31 02:48:11.144 751065 INFO cinder.volume.manager [req-5daf5729-9dc5-42e7-88c1-00a98cea8564 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Created volume successfully. >2018-08-31 02:48:17.170 751065 INFO cinder.volume.manager [req-139c1452-a6f3-4f89-8608-4f3f9ec76d3a cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Attaching volume 4c5385c7-2189-41f2-b549-16ec5b9240f0 to instance 62a69665-669f-4101-8cda-c92589db73dc at mountpoint /dev/vdb on host None. >2018-08-31 02:48:17.243 751065 INFO cinder.volume.manager [req-139c1452-a6f3-4f89-8608-4f3f9ec76d3a cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Attach volume completed successfully. >2018-08-31 02:48:17.857 751065 INFO cinder.volume.manager [req-aaec1a23-c46b-4abb-998f-5ea1f6c7a170 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Detaching volume 4c5385c7-2189-41f2-b549-16ec5b9240f0 from instance 62a69665-669f-4101-8cda-c92589db73dc. >2018-08-31 02:48:17.943 751065 INFO cinder.volume.manager [req-aaec1a23-c46b-4abb-998f-5ea1f6c7a170 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Detach volume completed successfully. >2018-08-31 02:48:25.407 751065 INFO cinder.volume.manager [req-4029917e-7ab8-4924-8e71-05f344bd17ec cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Attaching volume 4c5385c7-2189-41f2-b549-16ec5b9240f0 to instance f5fad6ed-4896-4664-9385-ffe8adc7a16a at mountpoint /dev/vdb on host None. >2018-08-31 02:48:25.477 751065 INFO cinder.volume.manager [req-4029917e-7ab8-4924-8e71-05f344bd17ec cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Attach volume completed successfully. >2018-08-31 02:48:26.193 751065 INFO cinder.volume.manager [req-9cfd0440-17a2-4261-aa29-c861a0aa34c9 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Detaching volume 4c5385c7-2189-41f2-b549-16ec5b9240f0 from instance f5fad6ed-4896-4664-9385-ffe8adc7a16a. >2018-08-31 02:48:26.275 751065 INFO cinder.volume.manager [req-9cfd0440-17a2-4261-aa29-c861a0aa34c9 cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Detach volume completed successfully. >2018-08-31 02:48:30.078 751065 INFO oslo.privsep.daemon [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpCeUVq9/privsep.sock'] >2018-08-31 02:48:31.096 751065 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-31 02:48:31.267 751065 INFO oslo.privsep.daemon [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Spawned new privsep daemon via rootwrap >2018-08-31 02:48:31.197 1042158 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-31 02:48:31.201 1042158 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-31 02:48:31.204 1042158 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-31 02:48:31.204 1042158 INFO oslo.privsep.daemon [-] privsep daemon running as pid 1042158 >2018-08-31 02:48:43.378 751065 INFO os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 02:50:43.986 751065 WARNING os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 02:50:43.989 751065 WARNING os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Failed to connect to iSCSI portal 11.11.109.102:3260. >2018-08-31 02:50:46.275 751065 INFO os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 02:52:46.865 751065 WARNING os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 02:52:46.868 751065 WARNING os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Failed to connect to iSCSI portal 11.11.109.102:3260.: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 02:52:51.149 751065 INFO os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 02:54:51.731 751065 WARNING os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 02:54:51.734 751065 WARNING os_brick.initiator.connectors.iscsi [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Failed to connect to iSCSI portal 11.11.109.102:3260.: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 02:54:58.144 751065 ERROR cinder.volume.manager [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Upload volume to image encountered an error (image-id: 3a0550ff-8485-402b-a184-7f871b33e15c).: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 02:54:58.145 751065 INFO cinder.message.api [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Creating message record for request_id = req-ff39e271-cf8d-46c1-a492-949f9b114e5f >2018-08-31 02:54:58.153 751065 WARNING oslo_config.cfg [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-31 02:54:58.488 751065 WARNING cinder.volume.manager [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Image delete encountered an error.: ImageNotFound: Image 3a0550ff-8485-402b-a184-7f871b33e15c could not be found. >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1532, in _delete_image >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager image_meta = image_service.show(context, image_id) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 268, in show >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager _reraise_translated_image_exception(image_id) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 582, in _reraise_translated_image_exception >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager six.reraise(type(new_exc), new_exc, exc_trace) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 266, in show >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager image = self._client.call(context, 'get', image_id) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 198, in call >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager return getattr(controller, method)(*args, **kwargs) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 197, in get >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager return self._get(image_id) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 545, in inner >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager return RequestIdProxy(wrapped(*args, **kwargs)) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 190, in _get >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager resp, body = self.http_client.get(url, headers=header) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 286, in get >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager return self._request('GET', url, **kwargs) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 278, in _request >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager resp, body_iter = self._handle_response(resp) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 98, in _handle_response >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager raise exc.from_response(resp, resp.content) >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager ImageNotFound: Image 3a0550ff-8485-402b-a184-7f871b33e15c could not be found. >2018-08-31 02:54:58.488 751065 ERROR cinder.volume.manager >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server [req-ff39e271-cf8d-46c1-a492-949f9b114e5f cbf3a161380d4c8f9bec7ade86a6377d bb674385e09f442db85b600c5f010cb9 - default default] Exception during message handling: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1522, in copy_volume_to_image >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server payload['message'] = six.text_type(error) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1503, in copy_volume_to_image >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server image_service, image_meta) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 850, in copy_volume_to_image >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server attach_info = self._connect_device(conn) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server device = connector.connect_volume(conn['data']) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 507, in connect_volume >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server self._cleanup_connection(connection_properties, force=True) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 501, in connect_volume >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server return self._connect_single_volume(connection_properties) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 61, in _wrapper >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server return r.call(f, *args, **kwargs) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server raise attempt.get() >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server six.reraise(self.value[0], self.value[1], self.value[2]) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server attempt = Attempt(fn(*args, **kwargs), attempt_number, False) >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 576, in _connect_single_volume >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server raise exception.VolumeDeviceNotFound(device='') >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server VolumeDeviceNotFound: Volume device not found at . >2018-08-31 02:54:58.540 751065 ERROR oslo_messaging.rpc.server >2018-08-31 02:58:38.104 751065 INFO cinder.volume.flows.manager.create_volume [req-164d84b1-f923-4204-93ad-a4633511c526 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Volume 0fd4b01a-885d-479b-be1f-e75951574b52: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0fd4b01a-885d-479b-be1f-e75951574b52'} >2018-08-31 02:58:38.757 751065 INFO cinder.volume.flows.manager.create_volume [req-164d84b1-f923-4204-93ad-a4633511c526 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Volume volume-0fd4b01a-885d-479b-be1f-e75951574b52 (0fd4b01a-885d-479b-be1f-e75951574b52): created successfully >2018-08-31 02:58:38.761 751065 INFO cinder.volume.manager [req-164d84b1-f923-4204-93ad-a4633511c526 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Created volume successfully. >2018-08-31 02:59:00.526 751065 INFO cinder.volume.manager [req-b3ff03f4-1590-4bc2-bb06-02c866ffecfe 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Initialize volume connection completed successfully. >2018-08-31 02:59:09.488 751065 INFO cinder.volume.manager [req-99297ad8-096d-44a6-a48e-d6ff1c33815b 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Attaching volume 0fd4b01a-885d-479b-be1f-e75951574b52 to instance 96469ead-1c00-4d4a-a5e4-54c93d8d7f85 at mountpoint /dev/vdb on host None. >2018-08-31 02:59:09.563 751065 INFO cinder.volume.manager [req-99297ad8-096d-44a6-a48e-d6ff1c33815b 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Attach volume completed successfully. >2018-08-31 02:59:11.128 751065 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a69b20dc-c579-47db-92c9-79848351b305 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Unable to get remote copy information for volume volume-0fd4b01a-885d-479b-be1f-e75951574b52 >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova [req-a69b20dc-c579-47db-92c9-79848351b305 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Failed to notify nova on events: [{'tag': '0fd4b01a-885d-479b-be1f-e75951574b52', 'name': 'volume-extended', 'server_uuid': '96469ead-1c00-4d4a-a5e4-54c93d8d7f85'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-605b5bb0-5d7e-4453-9584-f5519203d3ea) >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova return_raw=True) >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-605b5bb0-5d7e-4453-9584-f5519203d3ea) >2018-08-31 02:59:13.147 751065 ERROR cinder.compute.nova >2018-08-31 02:59:13.186 751065 INFO cinder.volume.manager [req-a69b20dc-c579-47db-92c9-79848351b305 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Extend volume completed successfully. >2018-08-31 03:04:22.895 751065 INFO cinder.volume.manager [req-5125abc7-5e66-4c26-a9c2-5daa612deb1f 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Terminate volume connection completed successfully. >2018-08-31 03:04:23.021 751065 INFO cinder.volume.manager [req-83f8fdff-c429-4ed8-9993-b824865c348f 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Detaching volume 0fd4b01a-885d-479b-be1f-e75951574b52 from instance 96469ead-1c00-4d4a-a5e4-54c93d8d7f85. >2018-08-31 03:04:23.107 751065 INFO cinder.volume.manager [req-83f8fdff-c429-4ed8-9993-b824865c348f 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Detach volume completed successfully. >2018-08-31 03:04:26.686 751065 INFO cinder.volume.manager [req-49379b4d-f280-478a-9ec0-f9893bb011cb 8e0d0e3462a9496eae927a1b4b6aafcd 21376dd85b8847a0b764464c8c29a3ce - default default] Deleted volume successfully. >2018-08-31 03:04:32.446 751065 INFO cinder.volume.flows.manager.create_volume [req-f84b1cfc-2a12-47a7-837a-9ea12298db29 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] Volume 0809d235-794d-4518-83c7-aa1ccc3ee539: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0809d235-794d-4518-83c7-aa1ccc3ee539'} >2018-08-31 03:04:33.036 751065 INFO cinder.volume.flows.manager.create_volume [req-f84b1cfc-2a12-47a7-837a-9ea12298db29 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] Volume volume-0809d235-794d-4518-83c7-aa1ccc3ee539 (0809d235-794d-4518-83c7-aa1ccc3ee539): created successfully >2018-08-31 03:04:33.040 751065 INFO cinder.volume.manager [req-f84b1cfc-2a12-47a7-837a-9ea12298db29 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] Created volume successfully. >2018-08-31 03:04:34.647 751065 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-54092d87-2780-4142-95ab-89d090c209c0 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] Unable to get remote copy information for volume volume-0809d235-794d-4518-83c7-aa1ccc3ee539 >2018-08-31 03:04:35.862 751065 INFO cinder.volume.manager [req-54092d87-2780-4142-95ab-89d090c209c0 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] Extend volume completed successfully. >2018-08-31 03:04:38.594 751065 INFO cinder.volume.manager [req-90be383a-e699-4201-9aa0-f754c3853405 5ba07087b2a443b182c7108e0a0e3cbe bcd0ff3ad7ed4584b5725281e98dfb62 - default default] Deleted volume successfully. >2018-08-31 03:04:43.098 751065 INFO cinder.volume.flows.manager.create_volume [req-0c36a48e-7839-4d97-a68b-13bb2411d882 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume e824a701-c9c7-4d5b-9b3e-ec0c1f0833c7: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-e824a701-c9c7-4d5b-9b3e-ec0c1f0833c7'} >2018-08-31 03:04:43.639 751065 INFO cinder.volume.flows.manager.create_volume [req-0c36a48e-7839-4d97-a68b-13bb2411d882 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume volume-e824a701-c9c7-4d5b-9b3e-ec0c1f0833c7 (e824a701-c9c7-4d5b-9b3e-ec0c1f0833c7): created successfully >2018-08-31 03:04:43.643 751065 INFO cinder.volume.manager [req-0c36a48e-7839-4d97-a68b-13bb2411d882 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Created volume successfully. >2018-08-31 03:04:46.084 751065 INFO cinder.volume.flows.manager.create_volume [req-9d201153-2b1b-4978-9f25-05db26da7abd 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume fe2cc7b3-dc99-4718-be89-b11e8b4f6d22: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fe2cc7b3-dc99-4718-be89-b11e8b4f6d22'} >2018-08-31 03:04:46.615 751065 INFO cinder.volume.flows.manager.create_volume [req-9d201153-2b1b-4978-9f25-05db26da7abd 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume volume-fe2cc7b3-dc99-4718-be89-b11e8b4f6d22 (fe2cc7b3-dc99-4718-be89-b11e8b4f6d22): created successfully >2018-08-31 03:04:46.619 751065 INFO cinder.volume.manager [req-9d201153-2b1b-4978-9f25-05db26da7abd 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Created volume successfully. >2018-08-31 03:04:58.902 751065 INFO cinder.volume.manager [req-bae97059-e983-442a-96d7-e019cbb59908 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Deleted volume successfully. >2018-08-31 03:05:00.961 751065 INFO cinder.volume.manager [req-c80616ab-81bc-4f8a-9af3-058b4aba268b 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Deleted volume successfully. >2018-08-31 03:05:01.279 751065 INFO cinder.volume.flows.manager.create_volume [req-e1c4e700-5c0e-4e76-ba39-f8a509c09bd9 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume 3b2c31c2-e036-4ead-885f-36a1e8dd3493: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3b2c31c2-e036-4ead-885f-36a1e8dd3493'} >2018-08-31 03:05:01.823 751065 INFO cinder.volume.flows.manager.create_volume [req-e1c4e700-5c0e-4e76-ba39-f8a509c09bd9 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume volume-3b2c31c2-e036-4ead-885f-36a1e8dd3493 (3b2c31c2-e036-4ead-885f-36a1e8dd3493): created successfully >2018-08-31 03:05:01.827 751065 INFO cinder.volume.manager [req-e1c4e700-5c0e-4e76-ba39-f8a509c09bd9 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Created volume successfully. >2018-08-31 03:05:02.697 751065 INFO cinder.volume.flows.manager.create_volume [req-86eeb3c7-8561-4967-a27f-c81ceb85255f 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume 12808b82-5ee2-45e4-bf0b-cf29c9b07eb3: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-12808b82-5ee2-45e4-bf0b-cf29c9b07eb3', 'source_volid': '3b2c31c2-e036-4ead-885f-36a1e8dd3493', 'source_volstatus': u'available'} >2018-08-31 03:05:14.675 751065 INFO cinder.volume.flows.manager.create_volume [req-86eeb3c7-8561-4967-a27f-c81ceb85255f 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume volume-12808b82-5ee2-45e4-bf0b-cf29c9b07eb3 (12808b82-5ee2-45e4-bf0b-cf29c9b07eb3): created successfully >2018-08-31 03:05:14.679 751065 INFO cinder.volume.manager [req-86eeb3c7-8561-4967-a27f-c81ceb85255f 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Created volume successfully. >2018-08-31 03:05:16.792 751065 INFO cinder.volume.flows.manager.create_volume [req-564fea95-bc13-435b-9563-b118cb875064 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume 83722793-73a6-4f7a-859f-8fa2c700c384: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-83722793-73a6-4f7a-859f-8fa2c700c384'} >2018-08-31 03:05:17.390 751065 INFO cinder.volume.flows.manager.create_volume [req-564fea95-bc13-435b-9563-b118cb875064 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume volume-83722793-73a6-4f7a-859f-8fa2c700c384 (83722793-73a6-4f7a-859f-8fa2c700c384): created successfully >2018-08-31 03:05:17.394 751065 INFO cinder.volume.manager [req-564fea95-bc13-435b-9563-b118cb875064 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Created volume successfully. >2018-08-31 03:05:24.545 751065 INFO cinder.volume.manager [req-bfc7ddc1-02ca-4525-82f1-d4968331bcb7 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Deleted volume successfully. >2018-08-31 03:05:28.834 751065 INFO cinder.volume.manager [req-4ee4cccf-e9aa-4a66-bbd2-40c66a897228 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Deleted volume successfully. >2018-08-31 03:05:30.177 751065 INFO cinder.volume.flows.manager.create_volume [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume 4cd18399-f09a-4f55-acb5-55b65e4706d2: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-4cd18399-f09a-4f55-acb5-55b65e4706d2', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f17a6ebe7d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 03:05:31.554 751065 INFO cinder.image.image_utils [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Image download 12.00 MB at 8.73 MB/s >2018-08-31 03:05:41.058 751065 INFO os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 03:07:42.645 751065 WARNING os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 03:07:42.647 751065 WARNING os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Failed to connect to iSCSI portal 11.11.109.102:3260. >2018-08-31 03:07:44.930 751065 INFO os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 03:09:45.530 751065 WARNING os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 03:09:45.534 751065 WARNING os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Failed to connect to iSCSI portal 11.11.109.102:3260.: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:09:49.821 751065 INFO os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 03:10:31.068 751065 INFO cinder.volume.manager [req-69703e8a-eb28-4c54-9252-46a7174e42b7 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Deleted volume successfully. >2018-08-31 03:10:40.095 751065 INFO cinder.volume.flows.manager.create_volume [req-6dc04f7c-8572-493d-92bc-577203a006bf a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Volume 5c4a167e-c86c-4862-8ac2-a1bf2bcfb8b9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5c4a167e-c86c-4862-8ac2-a1bf2bcfb8b9'} >2018-08-31 03:10:40.893 751065 INFO cinder.volume.flows.manager.create_volume [req-6dc04f7c-8572-493d-92bc-577203a006bf a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Volume volume-5c4a167e-c86c-4862-8ac2-a1bf2bcfb8b9 (5c4a167e-c86c-4862-8ac2-a1bf2bcfb8b9): created successfully >2018-08-31 03:10:40.896 751065 INFO cinder.volume.manager [req-6dc04f7c-8572-493d-92bc-577203a006bf a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Created volume successfully. >2018-08-31 03:10:42.464 751065 INFO cinder.volume.flows.manager.create_volume [req-8d430c45-249c-475e-a7f8-1f63126a1931 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Volume a66da446-3f79-4ab2-b08f-a2f364dddefe: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a66da446-3f79-4ab2-b08f-a2f364dddefe'} >2018-08-31 03:10:43.463 751065 INFO cinder.volume.flows.manager.create_volume [req-8d430c45-249c-475e-a7f8-1f63126a1931 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Volume volume-a66da446-3f79-4ab2-b08f-a2f364dddefe (a66da446-3f79-4ab2-b08f-a2f364dddefe): created successfully >2018-08-31 03:10:43.466 751065 INFO cinder.volume.manager [req-8d430c45-249c-475e-a7f8-1f63126a1931 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Created volume successfully. >2018-08-31 03:10:44.118 751049 INFO cinder.volume.flows.manager.create_volume [req-9e56d492-b54b-418f-a94f-a71ed00c0c40 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Volume cfaab687-06b6-4412-9d5d-9aea8108747f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cfaab687-06b6-4412-9d5d-9aea8108747f'} >2018-08-31 03:10:45.140 751049 INFO cinder.volume.flows.manager.create_volume [req-9e56d492-b54b-418f-a94f-a71ed00c0c40 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Volume volume-cfaab687-06b6-4412-9d5d-9aea8108747f (cfaab687-06b6-4412-9d5d-9aea8108747f): created successfully >2018-08-31 03:10:45.144 751049 INFO cinder.volume.manager [req-9e56d492-b54b-418f-a94f-a71ed00c0c40 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Created volume successfully. >2018-08-31 03:10:56.014 751065 INFO cinder.volume.manager [req-1b182eba-9098-447a-b765-de6d82fb0857 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Deleted volume successfully. >2018-08-31 03:10:56.033 751065 INFO cinder.volume.manager [req-63c3f01e-2185-4974-8c5e-84becf8679fb a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Deleted volume successfully. >2018-08-31 03:10:56.238 751049 INFO cinder.volume.manager [req-2ef05913-88f8-47f8-9eb6-8d959f55e290 a1d13adf948e46328a75a027f0ea62fb 110b73d9609c4dff90f44419492dba04 - default default] Deleted volume successfully. >2018-08-31 03:11:01.501 751065 INFO cinder.volume.flows.manager.create_volume [req-293761ff-5c2e-4b5f-8430-08486b7ab568 e7b8bf325d7142f097d8e2e9d80dd983 bbf047654c154f90942f584d5ad7bbc9 - default default] Volume fed4c1f9-7112-4ad9-b282-56229a62557e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fed4c1f9-7112-4ad9-b282-56229a62557e'} >2018-08-31 03:11:02.101 751065 INFO cinder.volume.flows.manager.create_volume [req-293761ff-5c2e-4b5f-8430-08486b7ab568 e7b8bf325d7142f097d8e2e9d80dd983 bbf047654c154f90942f584d5ad7bbc9 - default default] Volume volume-fed4c1f9-7112-4ad9-b282-56229a62557e (fed4c1f9-7112-4ad9-b282-56229a62557e): created successfully >2018-08-31 03:11:02.105 751065 INFO cinder.volume.manager [req-293761ff-5c2e-4b5f-8430-08486b7ab568 e7b8bf325d7142f097d8e2e9d80dd983 bbf047654c154f90942f584d5ad7bbc9 - default default] Created volume successfully. >2018-08-31 03:11:21.030 751065 INFO cinder.volume.manager [req-5e01dd7e-c24b-4ae7-95d5-8885c48ff2d2 e7b8bf325d7142f097d8e2e9d80dd983 bbf047654c154f90942f584d5ad7bbc9 - default default] Deleted volume successfully. >2018-08-31 03:11:25.401 751065 INFO cinder.volume.flows.manager.create_volume [req-db289f04-bdee-43a2-b982-de3919eb178d c900670717dc494298353b24ed8b54fc 6061641e4426404b8f3832f70b528803 - default default] Volume 6e22946e-3d7e-4e24-b556-05f75fb7a8de: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-6e22946e-3d7e-4e24-b556-05f75fb7a8de'} >2018-08-31 03:11:25.920 751065 INFO cinder.volume.flows.manager.create_volume [req-db289f04-bdee-43a2-b982-de3919eb178d c900670717dc494298353b24ed8b54fc 6061641e4426404b8f3832f70b528803 - default default] Volume volume-6e22946e-3d7e-4e24-b556-05f75fb7a8de (6e22946e-3d7e-4e24-b556-05f75fb7a8de): created successfully >2018-08-31 03:11:25.924 751065 INFO cinder.volume.manager [req-db289f04-bdee-43a2-b982-de3919eb178d c900670717dc494298353b24ed8b54fc 6061641e4426404b8f3832f70b528803 - default default] Created volume successfully. >2018-08-31 03:11:33.214 751065 INFO cinder.volume.manager [req-35d2ad18-69ee-49ff-a149-afef54f37e93 c900670717dc494298353b24ed8b54fc 6061641e4426404b8f3832f70b528803 - default default] Deleted volume successfully. >2018-08-31 03:11:50.405 751065 WARNING os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 03:11:50.407 751065 WARNING os_brick.initiator.connectors.iscsi [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Failed to connect to iSCSI portal 11.11.109.102:3260.: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Failed to copy image 6515695b-edb7-451c-870b-80a7ec9f72db to volume: 4cd18399-f09a-4f55-acb5-55b65e4706d2: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 555, in _copy_image_to_volume >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 791, in copy_image_to_volume >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, encrypted=False) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 811, in _copy_image_data_to_volume >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume attach_info, volume = self._attach_volume(context, volume, properties) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1000, in _attach_volume >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume attach_info = self._connect_device(conn) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1071, in _connect_device >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume device = connector.connect_volume(conn['data']) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume return f(*args, **kwargs) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 507, in connect_volume >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume self._cleanup_connection(connection_properties, force=True) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume self.force_reraise() >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume six.reraise(self.type_, self.value, self.tb) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 501, in connect_volume >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume return self._connect_single_volume(connection_properties) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 61, in _wrapper >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume return r.call(f, *args, **kwargs) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume raise attempt.get() >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume six.reraise(self.value[0], self.value[1], self.value[2]) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume attempt = Attempt(fn(*args, **kwargs), attempt_number, False) >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 576, in _connect_single_volume >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume raise exception.VolumeDeviceNotFound(device='') >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:11:52.317 751065 ERROR cinder.volume.flows.manager.create_volume >2018-08-31 03:11:52.349 751065 WARNING cinder.volume.manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e0a2b1cf-1a3f-4e44-954b-f165d077e83b) transitioned into state 'FAILURE' from state 'RUNNING' >5 predecessors (most recent first): > Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-31T03:05:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-174699022',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=4cd18399-f09a-4f55-acb5-55b65e4706d2,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='1d9fd73ddd4c474e83d7d2734f188bfe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-31T03:05:29Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-31T03:05:29Z,user_id='683757699ae5491caf7cb81c2115da49',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f17a6e7db90>}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-31T03:05:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-174699022',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=4cd18399-f09a-4f55-acb5-55b65e4706d2,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='1d9fd73ddd4c474e83d7d2734f188bfe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-31T03:05:29Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-31T03:05:29Z,user_id='683757699ae5491caf7cb81c2115da49',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(4cd18399-f09a-4f55-acb5-55b65e4706d2),volume_id=4cd18399-f09a-4f55-acb5-55b65e4706d2,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7f17a6e7db90>}, 'provides': {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'volume_id': '4cd18399-f09a-4f55-acb5-55b65e4706d2', 'volume_size': 1, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f17a6ebe7d0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}, 'volume_name': 'volume-4cd18399-f09a-4f55-acb5-55b65e4706d2', 'type': 'image'}} > |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-31T03:05:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-174699022',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=4cd18399-f09a-4f55-acb5-55b65e4706d2,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='1d9fd73ddd4c474e83d7d2734f188bfe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-31T03:05:29Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-31T03:05:29Z,user_id='683757699ae5491caf7cb81c2115da49',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'683757699ae5491caf7cb81c2115da49', u'availability_zone': u'nova', u'reservations': [u'39f682a6-79a4-4a14-99dc-d1845a655d06', u'4edd684e-b9d0-409c-bad1-bba57bfb5bcc'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {u'Type': u'Test'}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'1d9fd73ddd4c474e83d7d2734f188bfe', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesGetTest-Volume-174699022'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'683757699ae5491caf7cb81c2115da49', u'availability_zone': u'nova', u'reservations': [u'39f682a6-79a4-4a14-99dc-d1845a655d06', u'4edd684e-b9d0-409c-bad1-bba57bfb5bcc'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {u'Type': u'Test'}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'1d9fd73ddd4c474e83d7d2734f188bfe', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-VolumesGetTest-Volume-174699022'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'4cd18399-f09a-4f55-acb5-55b65e4706d2', u'size': 1, u'display_name': u'tempest-VolumesGetTest-Volume-174699022', u'display_description': None, u'cluster_name': None, u'name_id': u'4cd18399-f09a-4f55-acb5-55b65e4706d2', u'volume_admin_metadata': [], u'project_id': u'1d9fd73ddd4c474e83d7d2734f188bfe', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-4cd18399-f09a-4f55-acb5-55b65e4706d2', u'user_id': u'683757699ae5491caf7cb81c2115da49', u'bootable': False, u'created_at': u'2018-08-31T03:05:29.000000', u'attach_status': u'detached', u'volume_metadata': [{u'value': u'Test', u'key': u'Type'}], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {u'Type': u'Test'}}, u'image_id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'4cd18399-f09a-4f55-acb5-55b65e4706d2', u'group_id': None}, u'user_id': u'683757699ae5491caf7cb81c2115da49', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool'], u'hosts': [u'hostgroup@svc2#NonDR_DS8KBH00_Pool']}, u'metadata': {u'Type': u'Test'}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7f17a6e7db90>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=6515695b-edb7-451c-870b-80a7ec9f72db,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(4cd18399-f09a-4f55-acb5-55b65e4706d2),volume_id=4cd18399-f09a-4f55-acb5-55b65e4706d2,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None} > |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-31T03:05:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-174699022',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=4cd18399-f09a-4f55-acb5-55b65e4706d2,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='1d9fd73ddd4c474e83d7d2734f188bfe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-31T03:05:29Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-31T03:05:29Z,user_id='683757699ae5491caf7cb81c2115da49',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f17a6e7db90>}, 'provides': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-08-31T03:05:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-174699022',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@svc2#NonDR_DS8KBH00_Pool',id=4cd18399-f09a-4f55-acb5-55b65e4706d2,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='1d9fd73ddd4c474e83d7d2734f188bfe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-08-31T03:05:29Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='downloading',terminated_at=None,updated_at=2018-08-31T03:05:29Z,user_id='683757699ae5491caf7cb81c2115da49',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)} > |__Flow 'volume_create_manager': ImageCopyFailure: Failed to copy image to volume: Volume device not found at . >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager result = task.execute(**arguments) >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager **volume_spec) >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager image_service) >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager return f(*a, **k) >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager image_service >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager image_location, image_service) >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager raise exception.ImageCopyFailure(reason=ex) >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager ImageCopyFailure: Failed to copy image to volume: Volume device not found at . >2018-08-31 03:11:52.349 751065 ERROR cinder.volume.manager >2018-08-31 03:11:52.356 751065 WARNING cinder.volume.manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e0a2b1cf-1a3f-4e44-954b-f165d077e83b) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-31 03:11:52.359 751065 WARNING cinder.volume.manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (267290b3-81fe-467d-91cf-a8e02697355d) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-31 03:11:52.362 751065 WARNING cinder.volume.manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a6825ca7-2586-4217-8cbf-a1c9d19a46f1) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-31 03:11:52.369 751065 ERROR cinder.volume.flows.manager.create_volume [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Volume 4cd18399-f09a-4f55-acb5-55b65e4706d2: create failed >2018-08-31 03:11:52.370 751065 WARNING cinder.volume.manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6388410b-70c9-4f90-bf09-e098dec7c15f) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-31 03:11:52.373 751065 WARNING cinder.volume.manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2bcf9430-5c67-462f-b731-3aaeac3661b9) transitioned into state 'REVERTED' from state 'REVERTING' >2018-08-31 03:11:52.375 751065 WARNING cinder.volume.manager [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Flow 'volume_create_manager' (ab607a65-549b-49cb-9d9f-abaf94f6b9cc) transitioned into state 'REVERTED' from state 'RUNNING' >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server [req-641f4657-6bb0-4ab5-9131-83173ab15b61 683757699ae5491caf7cb81c2115da49 1d9fd73ddd4c474e83d7d2734f188bfe - default default] Exception during message handling: ImageCopyFailure: Failed to copy image to volume: Volume device not found at . >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server _run_flow() >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server flow_engine.run() >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server failures[0].reraise() >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 955, in execute >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server **volume_spec) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in _create_from_image >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server image_service) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 818, in _create_from_image_cache_or_download >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server image_service >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 698, in _create_from_image_download >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server image_location, image_service) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 575, in _copy_image_to_volume >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: Volume device not found at . >2018-08-31 03:11:52.380 751065 ERROR oslo_messaging.rpc.server >2018-08-31 03:12:29.808 751065 INFO cinder.volume.flows.manager.create_volume [req-57ade885-96cb-4234-8420-5fd40e2fd499 2f41d712ce8843c1bc8a9f3c6bb90384 99911b5a272847bd9b7b02218bfa90e3 - default default] Volume bdd95eb7-2142-4c3e-b603-29a18f7f69ec: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bdd95eb7-2142-4c3e-b603-29a18f7f69ec'} >2018-08-31 03:12:32.801 751065 INFO cinder.volume.flows.manager.create_volume [req-57ade885-96cb-4234-8420-5fd40e2fd499 2f41d712ce8843c1bc8a9f3c6bb90384 99911b5a272847bd9b7b02218bfa90e3 - default default] Volume volume-bdd95eb7-2142-4c3e-b603-29a18f7f69ec (bdd95eb7-2142-4c3e-b603-29a18f7f69ec): created successfully >2018-08-31 03:12:32.805 751065 INFO cinder.volume.manager [req-57ade885-96cb-4234-8420-5fd40e2fd499 2f41d712ce8843c1bc8a9f3c6bb90384 99911b5a272847bd9b7b02218bfa90e3 - default default] Created volume successfully. >2018-08-31 03:12:38.253 751065 INFO cinder.volume.manager [req-4f3b1250-bffd-4361-92b4-29e865ec3311 2f41d712ce8843c1bc8a9f3c6bb90384 99911b5a272847bd9b7b02218bfa90e3 - default default] Deleted volume successfully. >2018-08-31 03:12:44.583 751065 INFO cinder.volume.flows.manager.create_volume [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] Volume 5bb31544-78ea-4a2e-a373-cc8a006ab2fc: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-5bb31544-78ea-4a2e-a373-cc8a006ab2fc', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f17a6fc9350>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 03:12:45.441 751065 INFO cinder.image.image_utils [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-31 03:12:56.093 751065 INFO os_brick.initiator.connectors.iscsi [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] Trying to connect to iSCSI portal 9.11.110.191:3260 >2018-08-31 03:13:09.603 751065 WARNING os_brick.initiator.connectors.iscsi [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] LUN 0 on iSCSI portal 9.11.110.191:3260 not found on sysfs after logging in. >2018-08-31 03:13:11.885 751065 INFO os_brick.initiator.connectors.iscsi [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] Trying to connect to iSCSI portal 9.11.110.191:3260 >2018-08-31 03:20:24.929 751065 INFO cinder.image.image_utils [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] Converted 39.00 MB image at 0.09 MB/s >2018-08-31 03:20:31.664 751065 INFO cinder.volume.flows.manager.create_volume [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] Volume volume-5bb31544-78ea-4a2e-a373-cc8a006ab2fc (5bb31544-78ea-4a2e-a373-cc8a006ab2fc): created successfully >2018-08-31 03:20:31.668 751065 INFO cinder.volume.manager [req-6bc81e64-df72-47c3-898c-f7f86bf41df5 cc660dd02ab544a8a685fcaaecde7817 c676260c82584678a612bf2bc758d1fd - default default] Created volume successfully. >2018-08-31 03:22:49.523 751065 INFO cinder.volume.flows.manager.create_volume [req-d3fb1634-0db8-414b-8076-15dcd31895e1 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Volume 31bb14bf-1cda-45ae-84d3-d7a9f4eb4e8e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-31bb14bf-1cda-45ae-84d3-d7a9f4eb4e8e'} >2018-08-31 03:22:52.841 751065 INFO cinder.volume.flows.manager.create_volume [req-d3fb1634-0db8-414b-8076-15dcd31895e1 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Volume volume-31bb14bf-1cda-45ae-84d3-d7a9f4eb4e8e (31bb14bf-1cda-45ae-84d3-d7a9f4eb4e8e): created successfully >2018-08-31 03:22:52.846 751065 INFO cinder.volume.manager [req-d3fb1634-0db8-414b-8076-15dcd31895e1 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Created volume successfully. >2018-08-31 03:22:54.419 751049 INFO cinder.volume.flows.manager.create_volume [req-3197e874-52ea-4378-94ac-f90dc2df9d00 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Volume adf302b3-59e7-4a7f-bc15-73f4e467d383: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-adf302b3-59e7-4a7f-bc15-73f4e467d383'} >2018-08-31 03:22:54.933 751049 INFO cinder.volume.flows.manager.create_volume [req-3197e874-52ea-4378-94ac-f90dc2df9d00 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Volume volume-adf302b3-59e7-4a7f-bc15-73f4e467d383 (adf302b3-59e7-4a7f-bc15-73f4e467d383): created successfully >2018-08-31 03:22:54.937 751049 INFO cinder.volume.manager [req-3197e874-52ea-4378-94ac-f90dc2df9d00 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Created volume successfully. >2018-08-31 03:22:56.062 751065 INFO cinder.volume.flows.manager.create_volume [req-f6731de4-52d8-4f0d-aa55-6aba9d1e7921 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Volume 08ebb893-8e59-4922-8d4e-c3945170902f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-08ebb893-8e59-4922-8d4e-c3945170902f'} >2018-08-31 03:22:56.580 751065 INFO cinder.volume.flows.manager.create_volume [req-f6731de4-52d8-4f0d-aa55-6aba9d1e7921 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Volume volume-08ebb893-8e59-4922-8d4e-c3945170902f (08ebb893-8e59-4922-8d4e-c3945170902f): created successfully >2018-08-31 03:22:56.584 751065 INFO cinder.volume.manager [req-f6731de4-52d8-4f0d-aa55-6aba9d1e7921 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Created volume successfully. >2018-08-31 03:22:59.698 751065 INFO cinder.volume.manager [req-6d9ab296-af46-4030-8a37-8374018c97c2 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Deleted volume successfully. >2018-08-31 03:23:06.606 751049 INFO cinder.volume.manager [req-5bb478c1-4524-493c-acdd-a4f80be1c05f e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Deleted volume successfully. >2018-08-31 03:23:06.845 751065 INFO cinder.volume.manager [req-1d473d68-9ac3-4c62-a9d1-60ba3d04c199 e5a356c17a514b13a07b18e9bf2f1113 d030da7a102143cebfb97085605cda5f - default default] Deleted volume successfully. >2018-08-31 03:23:32.196 751049 INFO cinder.volume.flows.manager.create_volume [req-fd8a4574-573b-445b-b0de-6ed33f6aa65c 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Volume ec26c306-b525-45d2-aac7-634ac1f7777d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ec26c306-b525-45d2-aac7-634ac1f7777d'} >2018-08-31 03:23:32.766 751049 INFO cinder.volume.flows.manager.create_volume [req-fd8a4574-573b-445b-b0de-6ed33f6aa65c 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Volume volume-ec26c306-b525-45d2-aac7-634ac1f7777d (ec26c306-b525-45d2-aac7-634ac1f7777d): created successfully >2018-08-31 03:23:32.770 751049 INFO cinder.volume.manager [req-fd8a4574-573b-445b-b0de-6ed33f6aa65c 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Created volume successfully. >2018-08-31 03:23:34.440 751049 INFO cinder.rpc [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-31 03:23:34.445 751049 INFO cinder.rpc [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-31 03:23:34.596 751065 INFO cinder.volume.flows.manager.create_volume [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Volume 02ae6821-0bf6-4ccf-b656-87d569953c24: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-02ae6821-0bf6-4ccf-b656-87d569953c24'} >2018-08-31 03:23:35.811 751065 INFO cinder.volume.flows.manager.create_volume [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Volume volume-02ae6821-0bf6-4ccf-b656-87d569953c24 (02ae6821-0bf6-4ccf-b656-87d569953c24): created successfully >2018-08-31 03:23:35.815 751065 INFO cinder.volume.manager [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Created volume successfully. >2018-08-31 03:23:39.613 751049 INFO oslo.privsep.daemon [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmp85hl_P/privsep.sock'] >2018-08-31 03:23:40.422 751049 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-31 03:23:40.526 751049 INFO oslo.privsep.daemon [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Spawned new privsep daemon via rootwrap >2018-08-31 03:23:40.465 200596 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-31 03:23:40.468 200596 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-31 03:23:40.471 200596 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-31 03:23:40.471 200596 INFO oslo.privsep.daemon [-] privsep daemon running as pid 200596 >2018-08-31 03:23:52.349 751065 INFO cinder.volume.manager [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Initialize volume connection completed successfully. >2018-08-31 03:23:52.355 751049 INFO os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 03:25:52.930 751049 WARNING os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 03:25:52.933 751049 WARNING os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Failed to connect to iSCSI portal 11.11.109.102:3260. >2018-08-31 03:25:55.214 751049 INFO os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 03:27:55.804 751049 WARNING os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 03:27:55.807 751049 WARNING os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Failed to connect to iSCSI portal 11.11.109.102:3260.: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:28:00.084 751049 INFO os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Trying to connect to iSCSI portal 11.11.109.102:3260 >2018-08-31 03:30:00.658 751049 WARNING os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Failed to login iSCSI target iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 on portal 11.11.109.102:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command. >Command: iscsiadm -m node -T iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0 -p 11.11.109.102:3260 --login >Exit code: 8 >Stdout: u'Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260] (multiple)\n' >Stderr: u'iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.ibm:2145.arcsvc1.arc78cnft0, portal: 11.11.109.102,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n' >2018-08-31 03:30:00.661 751049 WARNING os_brick.initiator.connectors.iscsi [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Failed to connect to iSCSI portal 11.11.109.102:3260.: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Failed to copy volume ec26c306-b525-45d2-aac7-634ac1f7777d to 02ae6821-0bf6-4ccf-b656-87d569953c24: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2085, in _migrate_volume_generic >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager self._copy_volume_data(ctxt, volume, new_volume, remote='dest') >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1967, in _copy_volume_data >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager attach_encryptor=attach_encryptor) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1901, in _attach_volume >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager attach_info = self._connect_device(conn) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1865, in _connect_device >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager vol_handle = connector.connect_volume(conn['data']) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager return f(*args, **kwargs) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager return f(*args, **kwargs) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 507, in connect_volume >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager self._cleanup_connection(connection_properties, force=True) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager self.force_reraise() >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager six.reraise(self.type_, self.value, self.tb) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 501, in connect_volume >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager return self._connect_single_volume(connection_properties) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 61, in _wrapper >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager return r.call(f, *args, **kwargs) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager raise attempt.get() >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager six.reraise(self.value[0], self.value[1], self.value[2]) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager attempt = Attempt(fn(*args, **kwargs), attempt_number, False) >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 576, in _connect_single_volume >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager raise exception.VolumeDeviceNotFound(device='') >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:30:01.045 751049 ERROR cinder.volume.manager >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Exception during message handling: VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2745, in retype >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server new_reservations, status_update) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2741, in retype >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server new_type_id=new_type_id) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2348, in migrate_volume >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server volume.save() >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2341, in migrate_volume >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server self._migrate_volume_generic(ctxt, volume, host, new_type_id) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2107, in _migrate_volume_generic >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server new_volume) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2085, in _migrate_volume_generic >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server self._copy_volume_data(ctxt, volume, new_volume, remote='dest') >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1967, in _copy_volume_data >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server attach_encryptor=attach_encryptor) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1901, in _attach_volume >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server attach_info = self._connect_device(conn) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1865, in _connect_device >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server vol_handle = connector.connect_volume(conn['data']) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 507, in connect_volume >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server self._cleanup_connection(connection_properties, force=True) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 501, in connect_volume >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server return self._connect_single_volume(connection_properties) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 61, in _wrapper >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server return r.call(f, *args, **kwargs) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server raise attempt.get() >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server six.reraise(self.value[0], self.value[1], self.value[2]) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server attempt = Attempt(fn(*args, **kwargs), attempt_number, False) >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 576, in _connect_single_volume >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server raise exception.VolumeDeviceNotFound(device='') >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server VolumeDeviceNotFound: Volume device not found at . >2018-08-31 03:30:01.086 751049 ERROR oslo_messaging.rpc.server >2018-08-31 03:30:01.747 751065 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Error has occurred: Unexpected error while running command. >Command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-31 03:30:05.379 751065 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Error running SSH command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24": ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-31 03:30:05.382 751065 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-31 03:30:05.383 751065 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Error running SSH command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24": ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-31 03:30:05.383 751065 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] CLI Exception output: > command: ['svctask', 'rmvdisk', '"volume-02ae6821-0bf6-4ccf-b656-87d569953c24"'] > stdout: > stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask rmvdisk "volume-02ae6821-0bf6-4ccf-b656-87d569953c24" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings.\n' >2018-08-31 03:30:05.394 751065 ERROR cinder.volume.manager [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Unable to delete the destination volume during volume migration, (NOTE: database record needs to be deleted).: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'rmvdisk', '"volume-02ae6821-0bf6-4ccf-b656-87d569953c24"'] > stdout: > stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >. >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server [req-418529dc-502a-47bd-b967-75cc955944be 7c8919eb82264c25970c82389ed2ab36 16381259e48640728773dec1de4b32a8 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'rmvdisk', '"volume-02ae6821-0bf6-4ccf-b656-87d569953c24"'] > stdout: > stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >. >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 176, in _synchronized >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server return f(*a, **k) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "<string>", line 2, in delete_volume >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 793, in delete_volume >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server new_status) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server self.force_reraise() >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 775, in delete_volume >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server self.driver.delete_volume(volume) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2556, in delete_volume >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server self._helpers.delete_vdisk(volume['name'], False) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 1857, in delete_vdisk >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server self.ssh.rmvdisk(vdisk, force=force) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 458, in rmvdisk >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server self.run_ssh_assert_no_output(ssh_cmd) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 168, in run_ssh_assert_no_output >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server out, err = self._run_ssh(ssh_cmd) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 158, in _run_ssh >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=msg) >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server command: ['svctask', 'rmvdisk', '"volume-02ae6821-0bf6-4ccf-b656-87d569953c24"'] >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server stdout: >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server stderr: CMMVC8698E The command failed because the volume has host mappings, use -removehostmappings. >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server . >2018-08-31 03:30:05.395 751065 ERROR oslo_messaging.rpc.server >2018-08-31 03:35:17.925 751065 INFO cinder.volume.manager [req-d16b509a-addb-4761-b026-bc660fb86f1c e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Deleted volume successfully. >2018-08-31 03:37:36.977 750660 INFO oslo_service.service [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Caught SIGTERM, stopping children >2018-08-31 03:37:36.980 750660 INFO oslo_service.service [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Waiting on 2 children to exit >2018-08-31 03:37:44.029 750660 INFO oslo_service.service [req-60ae1a23-16e2-4b2e-a3c9-267272fc7b61 - - - - -] Child 751049 exited with status 0 >2018-08-31 03:38:36.998 751065 INFO oslo_service.service [-] Parent process has died unexpectedly, exiting >2018-08-31 03:38:41.506 287836 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports. >2018-08-31 03:38:43.357 287836 INFO cinder.rpc [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version. >2018-08-31 03:38:43.364 287836 INFO cinder.rpc [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version. >2018-08-31 03:38:43.377 287836 WARNING cinder.keymgr.conf_key_mgr [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] This key manager is insecure and is not recommended for production deployments >2018-08-31 03:38:44.488 287836 INFO cinder.volume.manager [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Image-volume cache disabled for host hostgroup@svc1. >2018-08-31 03:38:44.506 287836 INFO oslo_service.service [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Starting 1 workers >2018-08-31 03:38:44.517 288514 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-31 03:38:44.552 288514 INFO cinder.volume.manager [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Starting volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 03:38:44.568 288514 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 03:38:44.628 288514 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 03:38:45.538 287836 INFO cinder.volume.manager [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Image-volume cache disabled for host hostgroup@svc2. >2018-08-31 03:38:45.552 287836 INFO oslo_service.service [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Starting 1 workers >2018-08-31 03:38:45.562 287836 WARNING oslo_config.cfg [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-31 03:38:45.562 288753 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0) >2018-08-31 03:38:45.569 287836 WARNING oslo_config.cfg [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:45.569 287836 WARNING oslo_config.cfg [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:45.570 287836 WARNING oslo_config.cfg [req-6d516566-c3db-4d1a-a6ef-54e74a879076 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:45.577 288753 INFO cinder.volume.manager [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Starting volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 03:38:45.596 288753 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 03:38:45.664 288753 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 03:38:47.323 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] WWPN on node 252: [u'500507680C550D9E', u'500507680C140D9E', u'500507680C510D9E', u'500507680C220D9E', u'500507680C570D9E', u'500507680C240D9E', u'500507680C130D9E', u'500507680C270D9E', u'500507680C160D9E', u'500507680C530D9E', u'500507680C250D9E', u'500507680C280D9E', u'500507680C180D9E', u'500507680C210D9E', u'500507680C540D9E', u'500507680C150D9E', u'500507680C580D9E', u'500507680C170D9E', u'500507680C110D9E', u'500507680C560D9E', u'500507680C260D9E', u'500507680C520D9E', u'500507680C120D9E', u'500507680C230D9E']. >2018-08-31 03:38:47.725 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] WWPN on node 253: [u'500507680C550D9D', u'500507680C140D9D', u'500507680C510D9D', u'500507680C220D9D', u'500507680C570D9D', u'500507680C240D9D', u'500507680C130D9D', u'500507680C270D9D', u'500507680C160D9D', u'500507680C530D9D', u'500507680C250D9D', u'500507680C180D9D', u'500507680C280D9D', u'500507680C210D9D', u'500507680C540D9D', u'500507680C150D9D', u'500507680C580D9D', u'500507680C170D9D', u'500507680C110D9D', u'500507680C560D9D', u'500507680C520D9D', u'500507680C260D9D', u'500507680C120D9D', u'500507680C230D9D']. >2018-08-31 03:38:48.125 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] WWPN on node 254: [u'500507680C560DAD', u'500507680C150DAD', u'500507680C260DAD', u'500507680C510DAD', u'500507680C230DAD', u'500507680C110DAD', u'500507680C520DAD', u'500507680C540DAD', u'500507680C280DAD', u'500507680C220DAD', u'500507680C180DAD', u'500507680C250DAD', u'500507680C210DAD', u'500507680C120DAD', u'500507680C130DAD', u'500507680C160DAD', u'500507680C570DAD', u'500507680C240DAD', u'500507680C530DAD', u'500507680C580DAD', u'500507680C170DAD', u'500507680C140DAD', u'500507680C550DAD', u'500507680C270DAD']. >2018-08-31 03:38:48.536 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] WWPN on node 255: [u'500507680C560DAE', u'500507680C150DAE', u'500507680C260DAE', u'500507680C510DAE', u'500507680C230DAE', u'500507680C110DAE', u'500507680C520DAE', u'500507680C280DAE', u'500507680C540DAE', u'500507680C220DAE', u'500507680C180DAE', u'500507680C250DAE', u'500507680C210DAE', u'500507680C120DAE', u'500507680C130DAE', u'500507680C570DAE', u'500507680C160DAE', u'500507680C240DAE', u'500507680C530DAE', u'500507680C580DAE', u'500507680C170DAE', u'500507680C550DAE', u'500507680C140DAE', u'500507680C270DAE']. >2018-08-31 03:38:49.082 288753 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] WWPN on node 252: [u'500507680C550D9E', u'500507680C140D9E', u'500507680C510D9E', u'500507680C220D9E', u'500507680C570D9E', u'500507680C240D9E', u'500507680C130D9E', u'500507680C270D9E', u'500507680C160D9E', u'500507680C530D9E', u'500507680C250D9E', u'500507680C280D9E', u'500507680C180D9E', u'500507680C210D9E', u'500507680C540D9E', u'500507680C150D9E', u'500507680C580D9E', u'500507680C170D9E', u'500507680C110D9E', u'500507680C560D9E', u'500507680C260D9E', u'500507680C520D9E', u'500507680C120D9E', u'500507680C230D9E']. >2018-08-31 03:38:49.777 288753 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] WWPN on node 253: [u'500507680C550D9D', u'500507680C140D9D', u'500507680C510D9D', u'500507680C220D9D', u'500507680C570D9D', u'500507680C240D9D', u'500507680C130D9D', u'500507680C270D9D', u'500507680C160D9D', u'500507680C530D9D', u'500507680C250D9D', u'500507680C180D9D', u'500507680C280D9D', u'500507680C210D9D', u'500507680C540D9D', u'500507680C150D9D', u'500507680C580D9D', u'500507680C170D9D', u'500507680C110D9D', u'500507680C560D9D', u'500507680C520D9D', u'500507680C260D9D', u'500507680C120D9D', u'500507680C230D9D']. >2018-08-31 03:38:50.076 288514 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] ensure_export: Volume volume-e215809b-17c7-4f55-a533-faaa4177c25e not found on storage. >2018-08-31 03:38:50.159 288753 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] WWPN on node 254: [u'500507680C560DAD', u'500507680C150DAD', u'500507680C260DAD', u'500507680C510DAD', u'500507680C230DAD', u'500507680C110DAD', u'500507680C520DAD', u'500507680C540DAD', u'500507680C280DAD', u'500507680C220DAD', u'500507680C180DAD', u'500507680C250DAD', u'500507680C210DAD', u'500507680C120DAD', u'500507680C130DAD', u'500507680C160DAD', u'500507680C570DAD', u'500507680C240DAD', u'500507680C530DAD', u'500507680C580DAD', u'500507680C170DAD', u'500507680C140DAD', u'500507680C550DAD', u'500507680C270DAD']. >2018-08-31 03:38:50.490 288514 WARNING oslo_config.cfg [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:50.491 288514 WARNING oslo_config.cfg [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:50.492 288514 WARNING oslo_config.cfg [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:50.574 288753 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] WWPN on node 255: [u'500507680C560DAE', u'500507680C150DAE', u'500507680C260DAE', u'500507680C510DAE', u'500507680C230DAE', u'500507680C110DAE', u'500507680C520DAE', u'500507680C280DAE', u'500507680C540DAE', u'500507680C220DAE', u'500507680C180DAE', u'500507680C250DAE', u'500507680C210DAE', u'500507680C120DAE', u'500507680C130DAE', u'500507680C570DAE', u'500507680C160DAE', u'500507680C240DAE', u'500507680C530DAE', u'500507680C580DAE', u'500507680C170DAE', u'500507680C550DAE', u'500507680C140DAE', u'500507680C270DAE']. >2018-08-31 03:38:50.662 288514 INFO cinder.volume.manager [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Driver initialization completed successfully. >2018-08-31 03:38:50.671 288514 INFO cinder.manager [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Initiating service 6 cleanup >2018-08-31 03:38:50.674 288514 INFO cinder.manager [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Service 6 cleanup completed. >2018-08-31 03:38:50.954 288514 INFO cinder.volume.manager [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 03:38:51.321 288514 INFO cinder.volume.manager [req-fcd65bfc-3f68-403f-93e6-54d372fe5fb1 - - - - -] Driver post RPC initialization completed successfully. >2018-08-31 03:38:51.495 288753 WARNING oslo_config.cfg [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:51.496 288753 WARNING oslo_config.cfg [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:51.496 288753 WARNING oslo_config.cfg [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. >2018-08-31 03:38:51.630 288753 INFO cinder.volume.manager [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Driver initialization completed successfully. >2018-08-31 03:38:51.637 288753 INFO cinder.manager [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Initiating service 7 cleanup >2018-08-31 03:38:51.641 288753 INFO cinder.manager [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Service 7 cleanup completed. >2018-08-31 03:38:51.924 288753 INFO cinder.volume.manager [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Initializing RPC dependent components of volume driver StorwizeSVCISCSIDriver (2.2.1) >2018-08-31 03:38:52.333 288753 INFO cinder.volume.manager [req-aa1584b0-e1fa-4baa-94b3-791b386627d9 - - - - -] Driver post RPC initialization completed successfully. >2018-08-31 03:40:57.067 288514 INFO cinder.volume.flows.manager.create_volume [req-daec16d0-78bd-45ee-ab1d-80417f6e67a9 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume 2f1445f4-a609-460e-9cc0-7c1ee2076253: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2f1445f4-a609-460e-9cc0-7c1ee2076253'} >2018-08-31 03:40:58.534 288514 INFO cinder.volume.flows.manager.create_volume [req-daec16d0-78bd-45ee-ab1d-80417f6e67a9 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Volume volume-2f1445f4-a609-460e-9cc0-7c1ee2076253 (2f1445f4-a609-460e-9cc0-7c1ee2076253): created successfully >2018-08-31 03:40:58.538 288514 INFO cinder.volume.manager [req-daec16d0-78bd-45ee-ab1d-80417f6e67a9 e28902c88fac4c8fb262f733ea55c751 def303dd8d60408a807bf94835de5e0b - default default] Created volume successfully. >2018-08-31 03:48:11.874 288514 INFO cinder.volume.flows.manager.create_volume [req-757594b9-51b7-4145-823f-03950bf7c269 dddebe109a6a4e17abdc5e2dd2de9faa 551a67899c03498897fb7b5245913aee - default default] Volume 47eb2b26-a8fd-43a9-b7fa-361bf140aa57: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-47eb2b26-a8fd-43a9-b7fa-361bf140aa57'} >2018-08-31 03:48:13.216 288514 INFO cinder.volume.flows.manager.create_volume [req-757594b9-51b7-4145-823f-03950bf7c269 dddebe109a6a4e17abdc5e2dd2de9faa 551a67899c03498897fb7b5245913aee - default default] Volume volume-47eb2b26-a8fd-43a9-b7fa-361bf140aa57 (47eb2b26-a8fd-43a9-b7fa-361bf140aa57): created successfully >2018-08-31 03:48:13.220 288514 INFO cinder.volume.manager [req-757594b9-51b7-4145-823f-03950bf7c269 dddebe109a6a4e17abdc5e2dd2de9faa 551a67899c03498897fb7b5245913aee - default default] Created volume successfully. >2018-08-31 03:48:17.101 288514 INFO cinder.volume.manager [req-f1923263-1d2a-488d-a42c-f08955f353eb dddebe109a6a4e17abdc5e2dd2de9faa 551a67899c03498897fb7b5245913aee - default default] Deleted volume successfully. >2018-08-31 03:48:24.226 288514 INFO cinder.volume.flows.manager.create_volume [req-0288c7a2-0371-475d-99a3-b27247f9ac7a 3a1c25df263d42e9b7eaf72c234bec93 809cafeeefb9469f83e15ccdd475641c - default default] Volume ca2cf37d-9492-48d5-9869-e7a4ff283036: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ca2cf37d-9492-48d5-9869-e7a4ff283036'} >2018-08-31 03:48:25.672 288514 INFO cinder.volume.flows.manager.create_volume [req-0288c7a2-0371-475d-99a3-b27247f9ac7a 3a1c25df263d42e9b7eaf72c234bec93 809cafeeefb9469f83e15ccdd475641c - default default] Volume volume-ca2cf37d-9492-48d5-9869-e7a4ff283036 (ca2cf37d-9492-48d5-9869-e7a4ff283036): created successfully >2018-08-31 03:48:25.676 288514 INFO cinder.volume.manager [req-0288c7a2-0371-475d-99a3-b27247f9ac7a 3a1c25df263d42e9b7eaf72c234bec93 809cafeeefb9469f83e15ccdd475641c - default default] Created volume successfully. >2018-08-31 03:48:29.903 288514 WARNING cinder.quota [req-bd3416a4-ebc3-409e-ad6b-405e370a4da8 3a1c25df263d42e9b7eaf72c234bec93 809cafeeefb9469f83e15ccdd475641c - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1693897523 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1693897523, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 03:48:29.904 288514 WARNING cinder.quota [req-bd3416a4-ebc3-409e-ad6b-405e370a4da8 3a1c25df263d42e9b7eaf72c234bec93 809cafeeefb9469f83e15ccdd475641c - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1693897523 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1693897523, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 03:48:30.163 288514 INFO cinder.volume.manager [req-bd3416a4-ebc3-409e-ad6b-405e370a4da8 3a1c25df263d42e9b7eaf72c234bec93 809cafeeefb9469f83e15ccdd475641c - default default] Deleted volume successfully. >2018-08-31 03:49:17.302 288514 INFO cinder.volume.flows.manager.create_volume [req-620bb97e-7a9f-4162-b4da-280dabbb80aa cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume 504fed8d-0713-491c-8472-c35254aa48bb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-504fed8d-0713-491c-8472-c35254aa48bb'} >2018-08-31 03:49:18.722 288514 INFO cinder.volume.flows.manager.create_volume [req-620bb97e-7a9f-4162-b4da-280dabbb80aa cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume volume-504fed8d-0713-491c-8472-c35254aa48bb (504fed8d-0713-491c-8472-c35254aa48bb): created successfully >2018-08-31 03:49:18.727 288514 INFO cinder.volume.manager [req-620bb97e-7a9f-4162-b4da-280dabbb80aa cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Created volume successfully. >2018-08-31 03:49:19.159 288514 INFO cinder.volume.manager [req-ba13bc82-1d27-45bd-9232-9f9fae0fc839 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Attaching volume 504fed8d-0713-491c-8472-c35254aa48bb to instance e7b7a272-3c4f-4849-97b4-c60ac6f53e2d at mountpoint /dev/vdb on host None. >2018-08-31 03:49:19.241 288514 INFO cinder.volume.manager [req-ba13bc82-1d27-45bd-9232-9f9fae0fc839 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Attach volume completed successfully. >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager [req-8356d46e-dec0-46c1-b0e7-ca5ef65c5cb5 4338d55132fd4c92bd4cc31b7dd6b7f0 30d2481cae3744a1b379a1eb6452b3e5 - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager force=force) >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_iscsi.py", line 311, in terminate_connection >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-31 03:49:20.738 288514 ERROR cinder.volume.manager >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server [req-8356d46e-dec0-46c1-b0e7-ca5ef65c5cb5 4338d55132fd4c92bd4cc31b7dd6b7f0 30d2481cae3744a1b379a1eb6452b3e5 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-31 03:49:20.741 288514 ERROR oslo_messaging.rpc.server >2018-08-31 03:49:21.090 288514 INFO cinder.volume.manager [req-12d402ec-148a-492f-a675-1217cb682b5c cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Detaching volume 504fed8d-0713-491c-8472-c35254aa48bb from instance e7b7a272-3c4f-4849-97b4-c60ac6f53e2d. >2018-08-31 03:49:21.172 288514 INFO cinder.volume.manager [req-12d402ec-148a-492f-a675-1217cb682b5c cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Detach volume completed successfully. >2018-08-31 03:49:23.332 288753 INFO cinder.volume.flows.manager.create_volume [req-6ac8a9c8-8f2b-4a4f-920c-bb48db73da4f cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume c30fc765-0b82-4070-8316-133bfac759b3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c30fc765-0b82-4070-8316-133bfac759b3'} >2018-08-31 03:49:24.686 288753 INFO cinder.volume.flows.manager.create_volume [req-6ac8a9c8-8f2b-4a4f-920c-bb48db73da4f cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume volume-c30fc765-0b82-4070-8316-133bfac759b3 (c30fc765-0b82-4070-8316-133bfac759b3): created successfully >2018-08-31 03:49:24.690 288753 INFO cinder.volume.manager [req-6ac8a9c8-8f2b-4a4f-920c-bb48db73da4f cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Created volume successfully. >2018-08-31 03:49:28.870 288753 INFO cinder.volume.manager [req-de00d0fa-3067-42fd-b7e7-d2859ae4d30d 4338d55132fd4c92bd4cc31b7dd6b7f0 30d2481cae3744a1b379a1eb6452b3e5 - default default] Deleted volume successfully. >2018-08-31 03:49:29.215 288514 INFO cinder.volume.flows.manager.create_volume [req-539fe968-e6d9-4d71-a8d6-aa968fc2e24d cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume a7c65854-0a90-4099-bde1-60562c0bf7d5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a7c65854-0a90-4099-bde1-60562c0bf7d5'} >2018-08-31 03:49:30.695 288514 INFO cinder.volume.flows.manager.create_volume [req-539fe968-e6d9-4d71-a8d6-aa968fc2e24d cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume volume-a7c65854-0a90-4099-bde1-60562c0bf7d5 (a7c65854-0a90-4099-bde1-60562c0bf7d5): created successfully >2018-08-31 03:49:30.700 288514 INFO cinder.volume.manager [req-539fe968-e6d9-4d71-a8d6-aa968fc2e24d cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Created volume successfully. >2018-08-31 03:49:33.922 288514 INFO cinder.volume.manager [req-2114e6b7-5b29-459e-8dee-c527eab32f00 4338d55132fd4c92bd4cc31b7dd6b7f0 30d2481cae3744a1b379a1eb6452b3e5 - default default] Deleted volume successfully. >2018-08-31 03:49:34.751 288514 INFO cinder.volume.flows.manager.create_volume [req-4783fe7b-4c47-4d19-bae3-653b2a3da192 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume fe0a00ae-7648-4166-8881-f40354950807: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fe0a00ae-7648-4166-8881-f40354950807'} >2018-08-31 03:49:36.139 288514 INFO cinder.volume.flows.manager.create_volume [req-4783fe7b-4c47-4d19-bae3-653b2a3da192 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume volume-fe0a00ae-7648-4166-8881-f40354950807 (fe0a00ae-7648-4166-8881-f40354950807): created successfully >2018-08-31 03:49:36.143 288514 INFO cinder.volume.manager [req-4783fe7b-4c47-4d19-bae3-653b2a3da192 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Created volume successfully. >2018-08-31 03:49:39.162 288514 INFO cinder.volume.manager [req-a7609f4f-9769-4bb2-94bc-97fc3f681d5d 4338d55132fd4c92bd4cc31b7dd6b7f0 30d2481cae3744a1b379a1eb6452b3e5 - default default] Deleted volume successfully. >2018-08-31 03:49:39.620 288514 INFO cinder.volume.flows.manager.create_volume [req-cddbe6ec-87ab-42a6-b8fd-021501b246ab cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume 88fbc1a0-d6f4-4da8-8f73-2ef19d6deabd: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-88fbc1a0-d6f4-4da8-8f73-2ef19d6deabd'} >2018-08-31 03:49:41.053 288514 INFO cinder.volume.flows.manager.create_volume [req-cddbe6ec-87ab-42a6-b8fd-021501b246ab cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume volume-88fbc1a0-d6f4-4da8-8f73-2ef19d6deabd (88fbc1a0-d6f4-4da8-8f73-2ef19d6deabd): created successfully >2018-08-31 03:49:41.057 288514 INFO cinder.volume.manager [req-cddbe6ec-87ab-42a6-b8fd-021501b246ab cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Created volume successfully. >2018-08-31 03:49:43.869 288514 INFO cinder.volume.manager [req-d1c1ae28-2eeb-494f-ac4e-4b07620d585c 4338d55132fd4c92bd4cc31b7dd6b7f0 30d2481cae3744a1b379a1eb6452b3e5 - default default] Deleted volume successfully. >2018-08-31 03:49:44.452 288514 INFO cinder.volume.flows.manager.create_volume [req-8652a8a0-c2b4-4ef6-8ab7-3d0849715064 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume 80c0b00e-2d94-4b9d-b4be-3f04365e94d2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-80c0b00e-2d94-4b9d-b4be-3f04365e94d2'} >2018-08-31 03:49:46.163 288514 INFO cinder.volume.flows.manager.create_volume [req-8652a8a0-c2b4-4ef6-8ab7-3d0849715064 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Volume volume-80c0b00e-2d94-4b9d-b4be-3f04365e94d2 (80c0b00e-2d94-4b9d-b4be-3f04365e94d2): created successfully >2018-08-31 03:49:46.168 288514 INFO cinder.volume.manager [req-8652a8a0-c2b4-4ef6-8ab7-3d0849715064 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Created volume successfully. >2018-08-31 03:49:48.622 288514 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 03:49:48.694 288514 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 03:49:51.542 288514 INFO cinder.volume.manager [req-714cecdf-156a-405f-bac1-3c633c2874d2 cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Deleted volume successfully. >2018-08-31 03:49:51.809 288514 INFO cinder.volume.manager [req-8a97e935-916b-4838-86f1-c7091ec4ad1e cbb2dabd05b946a9a490805b81b11e02 f582b2777b384c7aa1b3a71bdcfb4c9f - default default] Deleted volume successfully. >2018-08-31 03:49:57.531 288514 INFO cinder.volume.flows.manager.create_volume [req-b7d9c01e-81f7-46f2-9b6a-cc966c450207 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Volume 5befc788-f693-4345-8003-f3b84604a462: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5befc788-f693-4345-8003-f3b84604a462'} >2018-08-31 03:49:59.309 288514 INFO cinder.volume.flows.manager.create_volume [req-b7d9c01e-81f7-46f2-9b6a-cc966c450207 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Volume volume-5befc788-f693-4345-8003-f3b84604a462 (5befc788-f693-4345-8003-f3b84604a462): created successfully >2018-08-31 03:49:59.314 288514 INFO cinder.volume.manager [req-b7d9c01e-81f7-46f2-9b6a-cc966c450207 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Created volume successfully. >2018-08-31 03:50:01.025 288514 INFO cinder.volume.flows.manager.create_volume [req-ccdc69d9-359d-4072-b73b-af0f77479ddf 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Volume a2d7ff8f-15fa-4b83-adbe-a6f4c0e3875e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a2d7ff8f-15fa-4b83-adbe-a6f4c0e3875e'} >2018-08-31 03:50:02.695 288514 INFO cinder.volume.flows.manager.create_volume [req-ccdc69d9-359d-4072-b73b-af0f77479ddf 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Volume volume-a2d7ff8f-15fa-4b83-adbe-a6f4c0e3875e (a2d7ff8f-15fa-4b83-adbe-a6f4c0e3875e): created successfully >2018-08-31 03:50:02.703 288514 INFO cinder.volume.manager [req-ccdc69d9-359d-4072-b73b-af0f77479ddf 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Created volume successfully. >2018-08-31 03:50:03.902 288753 INFO cinder.volume.flows.manager.create_volume [req-ba649fea-88fb-4e34-8f49-8b212f7ee46f 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Volume 508edc54-4efd-48db-8785-e9a98490cc57: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-508edc54-4efd-48db-8785-e9a98490cc57'} >2018-08-31 03:50:05.359 288753 INFO cinder.volume.flows.manager.create_volume [req-ba649fea-88fb-4e34-8f49-8b212f7ee46f 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Volume volume-508edc54-4efd-48db-8785-e9a98490cc57 (508edc54-4efd-48db-8785-e9a98490cc57): created successfully >2018-08-31 03:50:05.363 288753 INFO cinder.volume.manager [req-ba649fea-88fb-4e34-8f49-8b212f7ee46f 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Created volume successfully. >2018-08-31 03:50:06.733 288514 INFO cinder.volume.flows.manager.create_volume [req-55d5ecfb-b35c-4f74-acea-691539b93e35 584e9b5f81024d0a8119f661463b6050 43fb806c5bae4e938c8d77a3a629b168 - default default] Volume af4204ea-7603-4cc7-a755-9b5dcf43f3c9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-af4204ea-7603-4cc7-a755-9b5dcf43f3c9'} >2018-08-31 03:50:08.258 288514 INFO cinder.volume.flows.manager.create_volume [req-55d5ecfb-b35c-4f74-acea-691539b93e35 584e9b5f81024d0a8119f661463b6050 43fb806c5bae4e938c8d77a3a629b168 - default default] Volume volume-af4204ea-7603-4cc7-a755-9b5dcf43f3c9 (af4204ea-7603-4cc7-a755-9b5dcf43f3c9): created successfully >2018-08-31 03:50:08.262 288514 INFO cinder.volume.manager [req-55d5ecfb-b35c-4f74-acea-691539b93e35 584e9b5f81024d0a8119f661463b6050 43fb806c5bae4e938c8d77a3a629b168 - default default] Created volume successfully. >2018-08-31 03:50:10.615 288514 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 03:50:10.711 288514 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 03:50:16.842 288514 INFO cinder.volume.manager [req-33a7e951-3980-4ba1-a3c8-73fc34813c36 584e9b5f81024d0a8119f661463b6050 43fb806c5bae4e938c8d77a3a629b168 - default default] Deleted volume successfully. >2018-08-31 03:50:16.869 288514 INFO cinder.volume.manager [req-86763e3a-dd2c-4231-a590-2d25991fce18 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Deleted volume successfully. >2018-08-31 03:50:16.883 288753 INFO cinder.volume.manager [req-b868104c-411b-46ea-9043-1c3c7f5c5825 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Deleted volume successfully. >2018-08-31 03:50:16.995 288514 INFO cinder.volume.manager [req-9061583e-bb7b-4d6e-8be9-f768a3c98bc0 27cf8007aaef42219468597ed0e81569 e72c0bd71cc64bb28166c63660e43217 - default default] Deleted volume successfully. >2018-08-31 03:50:21.208 288514 INFO cinder.volume.flows.manager.create_volume [req-edea52d3-dca5-467f-bcee-19d7ad1aff66 4a3b351a86b5446f8b597aa0d0302ef2 abae4a988b514833a8b57fd2397c7457 - default default] Volume 8863ff71-715d-4c08-9e83-327b49fcc40e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8863ff71-715d-4c08-9e83-327b49fcc40e'} >2018-08-31 03:50:22.489 288514 INFO cinder.volume.flows.manager.create_volume [req-edea52d3-dca5-467f-bcee-19d7ad1aff66 4a3b351a86b5446f8b597aa0d0302ef2 abae4a988b514833a8b57fd2397c7457 - default default] Volume volume-8863ff71-715d-4c08-9e83-327b49fcc40e (8863ff71-715d-4c08-9e83-327b49fcc40e): created successfully >2018-08-31 03:50:22.493 288514 INFO cinder.volume.manager [req-edea52d3-dca5-467f-bcee-19d7ad1aff66 4a3b351a86b5446f8b597aa0d0302ef2 abae4a988b514833a8b57fd2397c7457 - default default] Created volume successfully. >2018-08-31 03:50:27.121 288514 INFO cinder.volume.manager [req-ca1d10f6-7ec9-43e0-98a0-0b4038c66b8e 4a3b351a86b5446f8b597aa0d0302ef2 abae4a988b514833a8b57fd2397c7457 - default default] Deleted volume successfully. >2018-08-31 03:50:33.824 288514 INFO cinder.volume.flows.manager.create_volume [req-08c87113-2850-402b-be87-583af3ae75a5 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] Volume ebaf6a88-2c46-42fd-89c4-339019b9b12e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ebaf6a88-2c46-42fd-89c4-339019b9b12e'} >2018-08-31 03:50:35.135 288514 INFO cinder.volume.flows.manager.create_volume [req-08c87113-2850-402b-be87-583af3ae75a5 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] Volume volume-ebaf6a88-2c46-42fd-89c4-339019b9b12e (ebaf6a88-2c46-42fd-89c4-339019b9b12e): created successfully >2018-08-31 03:50:35.141 288514 INFO cinder.volume.manager [req-08c87113-2850-402b-be87-583af3ae75a5 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] Created volume successfully. >2018-08-31 03:50:36.956 288514 INFO cinder.volume.manager [req-a861ccec-2bca-40e6-92c6-0ca77c85602e 677f8d8b83814b68bcd9958bb2e7566b 2ce7c1a07b224c52a03a845aab7ff340 - default default] Transfer volume completed successfully. >2018-08-31 03:50:39.905 288514 INFO cinder.volume.manager [req-ceabff18-0919-4ed7-89c9-3d2b2f3b7701 b4a399d94fb34b45af7716bcb1df9681 2d6b53290b544ec397bfe439cfdaf685 - default default] Deleted volume successfully. >2018-08-31 03:50:41.506 288514 INFO cinder.volume.flows.manager.create_volume [req-54bb6da4-d87e-4981-9442-3414b5b49437 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] Volume b27bdce2-852c-4bd3-94d7-2fa0c2ca70e1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b27bdce2-852c-4bd3-94d7-2fa0c2ca70e1'} >2018-08-31 03:50:42.920 288514 INFO cinder.volume.flows.manager.create_volume [req-54bb6da4-d87e-4981-9442-3414b5b49437 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] Volume volume-b27bdce2-852c-4bd3-94d7-2fa0c2ca70e1 (b27bdce2-852c-4bd3-94d7-2fa0c2ca70e1): created successfully >2018-08-31 03:50:42.924 288514 INFO cinder.volume.manager [req-54bb6da4-d87e-4981-9442-3414b5b49437 12dd42b6bc2546aca9d335409179249c 5906a7e9207f48e288ecc33e945f163c - default default] Created volume successfully. >2018-08-31 03:50:46.914 288514 INFO cinder.volume.manager [req-f6034929-1749-42da-91c4-cd078e7ca6f4 b4a399d94fb34b45af7716bcb1df9681 2d6b53290b544ec397bfe439cfdaf685 - default default] Deleted volume successfully. >2018-08-31 03:50:51.808 288514 INFO cinder.volume.flows.manager.create_volume [req-f94c09f1-f82e-4850-8974-7dd14b3e5d33 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Volume 62490a16-8b39-47ba-9c5b-c9453d261414: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-62490a16-8b39-47ba-9c5b-c9453d261414'} >2018-08-31 03:50:53.107 288514 INFO cinder.volume.flows.manager.create_volume [req-f94c09f1-f82e-4850-8974-7dd14b3e5d33 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Volume volume-62490a16-8b39-47ba-9c5b-c9453d261414 (62490a16-8b39-47ba-9c5b-c9453d261414): created successfully >2018-08-31 03:50:53.113 288514 INFO cinder.volume.manager [req-f94c09f1-f82e-4850-8974-7dd14b3e5d33 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Created volume successfully. >2018-08-31 03:51:00.628 288514 INFO cinder.volume.manager [req-af051064-90d5-473b-b304-a744f5b22644 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Attaching volume 62490a16-8b39-47ba-9c5b-c9453d261414 to instance d72ed3b0-a37b-4f65-836a-692f268b46a8 at mountpoint /dev/vdb on host None. >2018-08-31 03:51:00.700 288514 INFO cinder.volume.manager [req-af051064-90d5-473b-b304-a744f5b22644 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Attach volume completed successfully. >2018-08-31 03:51:01.127 288514 INFO cinder.volume.manager [req-db640954-52eb-4178-a624-56636103058a 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Detaching volume 62490a16-8b39-47ba-9c5b-c9453d261414 from instance d72ed3b0-a37b-4f65-836a-692f268b46a8. >2018-08-31 03:51:01.211 288514 INFO cinder.volume.manager [req-db640954-52eb-4178-a624-56636103058a 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Detach volume completed successfully. >2018-08-31 03:51:07.589 288514 INFO cinder.volume.manager [req-996ed9fa-f652-478c-8813-37fa437b06cd 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Attaching volume 62490a16-8b39-47ba-9c5b-c9453d261414 to instance 151d688b-3ac6-442f-9f1a-184b25d48fa9 at mountpoint /dev/vdb on host None. >2018-08-31 03:51:07.662 288514 INFO cinder.volume.manager [req-996ed9fa-f652-478c-8813-37fa437b06cd 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Attach volume completed successfully. >2018-08-31 03:51:08.566 288514 INFO cinder.volume.manager [req-4b02a1a0-e3fa-4323-bf71-792dd638f9e8 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Detaching volume 62490a16-8b39-47ba-9c5b-c9453d261414 from instance 151d688b-3ac6-442f-9f1a-184b25d48fa9. >2018-08-31 03:51:08.657 288514 INFO cinder.volume.manager [req-4b02a1a0-e3fa-4323-bf71-792dd638f9e8 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Detach volume completed successfully. >2018-08-31 03:51:12.860 288514 INFO oslo.privsep.daemon [req-2cdb9a2b-dd25-4fb1-a800-ca1135f942db 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpdTX1_V/privsep.sock'] >2018-08-31 03:51:13.683 288514 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-31 03:51:13.783 288514 INFO oslo.privsep.daemon [req-2cdb9a2b-dd25-4fb1-a800-ca1135f942db 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Spawned new privsep daemon via rootwrap >2018-08-31 03:51:13.727 361223 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-31 03:51:13.731 361223 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-31 03:51:13.733 361223 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-31 03:51:13.734 361223 INFO oslo.privsep.daemon [-] privsep daemon running as pid 361223 >2018-08-31 03:51:18.750 288514 INFO os_brick.initiator.connectors.iscsi [req-2cdb9a2b-dd25-4fb1-a800-ca1135f942db 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 03:51:23.712 288514 WARNING oslo_config.cfg [req-2cdb9a2b-dd25-4fb1-a800-ca1135f942db 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-31 03:51:45.979 288514 INFO cinder.volume.manager [req-2cdb9a2b-dd25-4fb1-a800-ca1135f942db 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Copy volume to image completed successfully. >2018-08-31 03:51:49.567 288514 INFO cinder.volume.manager [req-c704154f-6a05-464e-9b43-70d1c8cb8832 8e7a5aca1a2344d6bc2c91a382630a72 e0936e753bfb4e2bac011f10cef73006 - default default] Deleted volume successfully. >2018-08-31 03:51:55.301 288514 INFO cinder.volume.flows.manager.create_volume [req-999ba08e-8090-4ea0-b9e6-ef941628d232 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Volume c027ec2f-1ce6-46f3-a130-333c9c8e7255: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c027ec2f-1ce6-46f3-a130-333c9c8e7255'} >2018-08-31 03:51:56.615 288514 INFO cinder.volume.flows.manager.create_volume [req-999ba08e-8090-4ea0-b9e6-ef941628d232 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Volume volume-c027ec2f-1ce6-46f3-a130-333c9c8e7255 (c027ec2f-1ce6-46f3-a130-333c9c8e7255): created successfully >2018-08-31 03:51:56.619 288514 INFO cinder.volume.manager [req-999ba08e-8090-4ea0-b9e6-ef941628d232 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Created volume successfully. >2018-08-31 03:52:12.446 288514 INFO cinder.volume.manager [req-f5fabc72-7677-4a47-8971-7cae910069ea 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Initialize volume connection completed successfully. >2018-08-31 03:52:21.344 288514 INFO cinder.volume.manager [req-4f621fc0-a260-4811-bd8e-86fe713c8fc6 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Attaching volume c027ec2f-1ce6-46f3-a130-333c9c8e7255 to instance cf3591ea-a7af-44f1-89cd-f4ef50abf351 at mountpoint /dev/vdb on host None. >2018-08-31 03:52:21.648 288514 INFO cinder.volume.manager [req-4f621fc0-a260-4811-bd8e-86fe713c8fc6 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Attach volume completed successfully. >2018-08-31 03:52:23.342 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-f8ae8a6e-68c3-431a-a6cd-dba5b4281cf0 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Unable to get remote copy information for volume volume-c027ec2f-1ce6-46f3-a130-333c9c8e7255 >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova [req-f8ae8a6e-68c3-431a-a6cd-dba5b4281cf0 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Failed to notify nova on events: [{'tag': 'c027ec2f-1ce6-46f3-a130-333c9c8e7255', 'name': 'volume-extended', 'server_uuid': 'cf3591ea-a7af-44f1-89cd-f4ef50abf351'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-3076f6cd-a1f2-4976-9a6d-cbd15edb2427) >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova return_raw=True) >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-3076f6cd-a1f2-4976-9a6d-cbd15edb2427) >2018-08-31 03:52:25.082 288514 ERROR cinder.compute.nova >2018-08-31 03:52:25.121 288514 INFO cinder.volume.manager [req-f8ae8a6e-68c3-431a-a6cd-dba5b4281cf0 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Extend volume completed successfully. >2018-08-31 03:57:35.955 288514 INFO cinder.volume.manager [req-3e2db47e-8d4a-477e-8ca6-6ddc7ec28748 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Terminate volume connection completed successfully. >2018-08-31 03:57:36.320 288514 INFO cinder.volume.manager [req-afd0ca17-f82d-447a-a385-0b7b08d4c0af 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Detaching volume c027ec2f-1ce6-46f3-a130-333c9c8e7255 from instance cf3591ea-a7af-44f1-89cd-f4ef50abf351. >2018-08-31 03:57:36.428 288514 INFO cinder.volume.manager [req-afd0ca17-f82d-447a-a385-0b7b08d4c0af 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Detach volume completed successfully. >2018-08-31 03:57:40.845 288514 INFO cinder.volume.manager [req-36185fc3-8555-4581-9407-43b623ef97dc 4b4b996d014d44ef925dfe70415a3c71 a4fc9fb00e0e4ad38717d317a235355a - default default] Deleted volume successfully. >2018-08-31 03:57:45.305 288514 INFO cinder.volume.flows.manager.create_volume [req-87e115c5-3d60-4c59-b5ed-257f7359c468 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] Volume 7de3d53f-0732-4cb0-83ba-ff536e07e69b: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7de3d53f-0732-4cb0-83ba-ff536e07e69b'} >2018-08-31 03:57:46.618 288514 INFO cinder.volume.flows.manager.create_volume [req-87e115c5-3d60-4c59-b5ed-257f7359c468 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] Volume volume-7de3d53f-0732-4cb0-83ba-ff536e07e69b (7de3d53f-0732-4cb0-83ba-ff536e07e69b): created successfully >2018-08-31 03:57:46.622 288514 INFO cinder.volume.manager [req-87e115c5-3d60-4c59-b5ed-257f7359c468 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] Created volume successfully. >2018-08-31 03:57:47.484 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-24087d40-3a7b-4734-a0b5-a0f6896992a0 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] Unable to get remote copy information for volume volume-7de3d53f-0732-4cb0-83ba-ff536e07e69b >2018-08-31 03:57:48.527 288514 INFO cinder.volume.manager [req-24087d40-3a7b-4734-a0b5-a0f6896992a0 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] Extend volume completed successfully. >2018-08-31 03:57:51.153 288514 INFO cinder.volume.manager [req-904f5dc8-8da0-45ac-8162-f59f5bacdfdf 7c63614f745d4748a57ce44f4e10af29 8aaef12a7cfc4a8cb28158a75a9169fd - default default] Deleted volume successfully. >2018-08-31 03:57:54.870 288514 INFO cinder.volume.flows.manager.create_volume [req-81b75b39-0ad3-44ca-b553-1fc614df7dbc 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume 4be59e95-49ed-4603-8b58-d1cd4fca7e70: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4be59e95-49ed-4603-8b58-d1cd4fca7e70'} >2018-08-31 03:57:56.157 288514 INFO cinder.volume.flows.manager.create_volume [req-81b75b39-0ad3-44ca-b553-1fc614df7dbc 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume volume-4be59e95-49ed-4603-8b58-d1cd4fca7e70 (4be59e95-49ed-4603-8b58-d1cd4fca7e70): created successfully >2018-08-31 03:57:56.161 288514 INFO cinder.volume.manager [req-81b75b39-0ad3-44ca-b553-1fc614df7dbc 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Created volume successfully. >2018-08-31 03:57:58.035 288514 INFO cinder.volume.flows.manager.create_volume [req-8d28cb60-14bc-4ce9-bce1-0157c0a43a56 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume b6287d53-2da7-4b20-b3ab-16ae8ca598cb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b6287d53-2da7-4b20-b3ab-16ae8ca598cb'} >2018-08-31 03:57:59.429 288514 INFO cinder.volume.flows.manager.create_volume [req-8d28cb60-14bc-4ce9-bce1-0157c0a43a56 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume volume-b6287d53-2da7-4b20-b3ab-16ae8ca598cb (b6287d53-2da7-4b20-b3ab-16ae8ca598cb): created successfully >2018-08-31 03:57:59.433 288514 INFO cinder.volume.manager [req-8d28cb60-14bc-4ce9-bce1-0157c0a43a56 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Created volume successfully. >2018-08-31 03:58:02.865 288514 INFO cinder.volume.manager [req-705a5cef-ccbe-4ed6-9874-6fe5e14f071a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Deleted volume successfully. >2018-08-31 03:58:05.903 288514 INFO cinder.volume.manager [req-68d89c03-5f28-4e2c-9e0f-e42672fe4cf7 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Deleted volume successfully. >2018-08-31 03:58:06.704 288514 INFO cinder.volume.flows.manager.create_volume [req-6630413a-3f5a-4b0f-8744-d6fb3025e15d 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume f7cd4f49-c2ce-437a-919f-122898fbc192: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f7cd4f49-c2ce-437a-919f-122898fbc192'} >2018-08-31 03:58:08.509 288514 INFO cinder.volume.flows.manager.create_volume [req-6630413a-3f5a-4b0f-8744-d6fb3025e15d 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume volume-f7cd4f49-c2ce-437a-919f-122898fbc192 (f7cd4f49-c2ce-437a-919f-122898fbc192): created successfully >2018-08-31 03:58:08.513 288514 INFO cinder.volume.manager [req-6630413a-3f5a-4b0f-8744-d6fb3025e15d 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Created volume successfully. >2018-08-31 03:58:09.227 288514 INFO cinder.volume.flows.manager.create_volume [req-a350314e-06ff-4b13-8929-70b081fda44d 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume 7e4cb048-df70-4cd5-9966-049a563a9a85: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7e4cb048-df70-4cd5-9966-049a563a9a85', 'source_volid': 'f7cd4f49-c2ce-437a-919f-122898fbc192', 'source_volstatus': u'available'} >2018-08-31 03:58:11.752 288514 INFO cinder.volume.flows.manager.create_volume [req-a350314e-06ff-4b13-8929-70b081fda44d 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume volume-7e4cb048-df70-4cd5-9966-049a563a9a85 (7e4cb048-df70-4cd5-9966-049a563a9a85): created successfully >2018-08-31 03:58:11.757 288514 INFO cinder.volume.manager [req-a350314e-06ff-4b13-8929-70b081fda44d 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Created volume successfully. >2018-08-31 03:58:13.560 288514 INFO cinder.volume.flows.manager.create_volume [req-2f5efa16-1e0a-4e8c-bbd3-2f12723efbb9 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume bdd2970c-9b1e-462c-9338-1bb031ab76c6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bdd2970c-9b1e-462c-9338-1bb031ab76c6'} >2018-08-31 03:58:15.159 288514 INFO cinder.volume.flows.manager.create_volume [req-2f5efa16-1e0a-4e8c-bbd3-2f12723efbb9 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume volume-bdd2970c-9b1e-462c-9338-1bb031ab76c6 (bdd2970c-9b1e-462c-9338-1bb031ab76c6): created successfully >2018-08-31 03:58:15.164 288514 INFO cinder.volume.manager [req-2f5efa16-1e0a-4e8c-bbd3-2f12723efbb9 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Created volume successfully. >2018-08-31 03:58:17.927 288514 INFO cinder.volume.manager [req-d2fcbbf2-c655-4936-aed3-882ee4d678c6 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Deleted volume successfully. >2018-08-31 03:58:23.280 288514 INFO cinder.volume.manager [req-2f4c322a-53fd-4812-b848-2fb4f25ea986 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Deleted volume successfully. >2018-08-31 03:58:25.462 288514 INFO cinder.volume.flows.manager.create_volume [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume fd7b6e85-f6d3-45df-974e-151103b0977e: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-fd7b6e85-f6d3-45df-974e-151103b0977e', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f636d02a110>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 03:58:27.018 288514 INFO cinder.image.image_utils [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Image download 12.00 MB at 7.73 MB/s >2018-08-31 03:58:34.291 288514 INFO os_brick.initiator.connectors.iscsi [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Trying to connect to iSCSI portal 9.11.54.170:3260 >2018-08-31 03:58:47.804 288514 WARNING os_brick.initiator.connectors.iscsi [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] LUN 0 on iSCSI portal 9.11.54.170:3260 not found on sysfs after logging in. >2018-08-31 03:58:50.096 288514 INFO os_brick.initiator.connectors.iscsi [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Trying to connect to iSCSI portal 9.11.54.170:3260 >2018-08-31 03:58:58.013 288514 INFO cinder.image.image_utils [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Converted 39.00 MB image at 36.23 MB/s >2018-08-31 03:59:00.657 288514 INFO cinder.volume.flows.manager.create_volume [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume volume-fd7b6e85-f6d3-45df-974e-151103b0977e (fd7b6e85-f6d3-45df-974e-151103b0977e): created successfully >2018-08-31 03:59:00.663 288514 INFO cinder.volume.manager [req-1758ea1a-5fb3-4842-8f93-a5be5e9a728a 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Created volume successfully. >2018-08-31 03:59:02.472 288514 INFO cinder.volume.flows.manager.create_volume [req-501ff8e3-4caf-4d52-84fc-6c2c7749ef34 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume 4dc322aa-8ddf-4142-9f72-438ddc432625: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4dc322aa-8ddf-4142-9f72-438ddc432625'} >2018-08-31 03:59:04.067 288514 INFO cinder.volume.flows.manager.create_volume [req-501ff8e3-4caf-4d52-84fc-6c2c7749ef34 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Volume volume-4dc322aa-8ddf-4142-9f72-438ddc432625 (4dc322aa-8ddf-4142-9f72-438ddc432625): created successfully >2018-08-31 03:59:04.072 288514 INFO cinder.volume.manager [req-501ff8e3-4caf-4d52-84fc-6c2c7749ef34 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Created volume successfully. >2018-08-31 03:59:06.697 288514 INFO cinder.volume.manager [req-b2bcfa3f-bbba-48c6-aa70-97f8073d39b3 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Deleted volume successfully. >2018-08-31 03:59:09.171 288514 INFO cinder.volume.manager [req-e8c68e46-5056-47d3-ad46-cd8f6b7d6ca0 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Deleted volume successfully. >2018-08-31 03:59:11.549 288514 INFO cinder.volume.manager [req-0182fe23-1e18-49c5-940b-d19552c918fa 520b3aa20641451088dbe4470a4b4567 827465256fa843ecb1d4a060cfe6fd83 - default default] Deleted volume successfully. >2018-08-31 03:59:18.286 288514 INFO cinder.volume.flows.manager.create_volume [req-f54c6ab6-f21f-44e4-a65e-31deda19793c 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Volume 8fe727e6-8f90-4796-acb4-cafd71a23653: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8fe727e6-8f90-4796-acb4-cafd71a23653'} >2018-08-31 03:59:19.567 288514 INFO cinder.volume.flows.manager.create_volume [req-f54c6ab6-f21f-44e4-a65e-31deda19793c 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Volume volume-8fe727e6-8f90-4796-acb4-cafd71a23653 (8fe727e6-8f90-4796-acb4-cafd71a23653): created successfully >2018-08-31 03:59:19.571 288514 INFO cinder.volume.manager [req-f54c6ab6-f21f-44e4-a65e-31deda19793c 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Created volume successfully. >2018-08-31 03:59:21.749 288514 INFO cinder.volume.flows.manager.create_volume [req-0a19a0cf-c83b-40f4-ab44-189c64d06fc2 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Volume 4a8cf9b0-bacc-41e4-819d-ab700393c363: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-4a8cf9b0-bacc-41e4-819d-ab700393c363'} >2018-08-31 03:59:23.060 288514 INFO cinder.volume.flows.manager.create_volume [req-0a19a0cf-c83b-40f4-ab44-189c64d06fc2 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Volume volume-4a8cf9b0-bacc-41e4-819d-ab700393c363 (4a8cf9b0-bacc-41e4-819d-ab700393c363): created successfully >2018-08-31 03:59:23.064 288514 INFO cinder.volume.manager [req-0a19a0cf-c83b-40f4-ab44-189c64d06fc2 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Created volume successfully. >2018-08-31 03:59:23.826 288514 INFO cinder.volume.flows.manager.create_volume [req-468f6703-6780-4425-b5bd-23cf723b43b0 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Volume eeb9e191-95b8-4392-815d-55004cb0dd40: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-eeb9e191-95b8-4392-815d-55004cb0dd40'} >2018-08-31 03:59:25.112 288514 INFO cinder.volume.flows.manager.create_volume [req-468f6703-6780-4425-b5bd-23cf723b43b0 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Volume volume-eeb9e191-95b8-4392-815d-55004cb0dd40 (eeb9e191-95b8-4392-815d-55004cb0dd40): created successfully >2018-08-31 03:59:25.117 288514 INFO cinder.volume.manager [req-468f6703-6780-4425-b5bd-23cf723b43b0 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Created volume successfully. >2018-08-31 03:59:33.898 288514 INFO cinder.volume.manager [req-f35b15f7-0ae0-483e-809d-c07d0db6cb26 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Deleted volume successfully. >2018-08-31 03:59:33.917 288514 INFO cinder.volume.manager [req-05af124b-0590-426b-aa7a-490e44aedd42 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Deleted volume successfully. >2018-08-31 03:59:34.148 288514 INFO cinder.volume.manager [req-73c8c28a-4238-4368-870b-8fe8ff6ffdc5 5fb0ea0ab33d49a589a7c92ff9f243ec dd748d6b54024918bd29aa399fa30385 - default default] Deleted volume successfully. >2018-08-31 03:59:37.831 288514 INFO cinder.volume.flows.manager.create_volume [req-aa013e4c-6b2f-4974-a46c-e7c7209056bb 6a411231f618432d893d28153458c01d 786c65e102c54ff686b3f678d58a5367 - default default] Volume d00b65e2-fd97-43b2-8588-923138431fdb: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d00b65e2-fd97-43b2-8588-923138431fdb'} >2018-08-31 03:59:39.198 288514 INFO cinder.volume.flows.manager.create_volume [req-aa013e4c-6b2f-4974-a46c-e7c7209056bb 6a411231f618432d893d28153458c01d 786c65e102c54ff686b3f678d58a5367 - default default] Volume volume-d00b65e2-fd97-43b2-8588-923138431fdb (d00b65e2-fd97-43b2-8588-923138431fdb): created successfully >2018-08-31 03:59:39.202 288514 INFO cinder.volume.manager [req-aa013e4c-6b2f-4974-a46c-e7c7209056bb 6a411231f618432d893d28153458c01d 786c65e102c54ff686b3f678d58a5367 - default default] Created volume successfully. >2018-08-31 04:00:01.271 288514 INFO cinder.volume.manager [req-e4db8a53-95fb-49dd-b172-85264f08c075 6a411231f618432d893d28153458c01d 786c65e102c54ff686b3f678d58a5367 - default default] Deleted volume successfully. >2018-08-31 04:00:04.522 288514 INFO cinder.volume.flows.manager.create_volume [req-0877a527-11cc-4bc8-86dd-48928f34dc3d 1b8bca24f4a34b778ad519a02cc8919b ca1628f688d84a5f99c044a74fff2f84 - default default] Volume c04a9a68-ad31-4015-905e-2b97553ffbea: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c04a9a68-ad31-4015-905e-2b97553ffbea'} >2018-08-31 04:00:05.840 288514 INFO cinder.volume.flows.manager.create_volume [req-0877a527-11cc-4bc8-86dd-48928f34dc3d 1b8bca24f4a34b778ad519a02cc8919b ca1628f688d84a5f99c044a74fff2f84 - default default] Volume volume-c04a9a68-ad31-4015-905e-2b97553ffbea (c04a9a68-ad31-4015-905e-2b97553ffbea): created successfully >2018-08-31 04:00:05.845 288514 INFO cinder.volume.manager [req-0877a527-11cc-4bc8-86dd-48928f34dc3d 1b8bca24f4a34b778ad519a02cc8919b ca1628f688d84a5f99c044a74fff2f84 - default default] Created volume successfully. >2018-08-31 04:00:08.773 288514 INFO cinder.volume.manager [req-886d47e9-7aa7-467c-bee6-fe20b53a501d 1b8bca24f4a34b778ad519a02cc8919b ca1628f688d84a5f99c044a74fff2f84 - default default] Deleted volume successfully. >2018-08-31 04:00:56.374 288514 INFO cinder.volume.flows.manager.create_volume [req-f6e0244c-b398-4fe3-87e5-18219ca2fe0b 64fc12ed066b494e95b98d115be08441 ea69b8db42db4aeb99c9fefc06dcca53 - default default] Volume ed275f36-fb43-4825-aa59-5b3b0da5c62f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ed275f36-fb43-4825-aa59-5b3b0da5c62f'} >2018-08-31 04:00:57.652 288514 INFO cinder.volume.flows.manager.create_volume [req-f6e0244c-b398-4fe3-87e5-18219ca2fe0b 64fc12ed066b494e95b98d115be08441 ea69b8db42db4aeb99c9fefc06dcca53 - default default] Volume volume-ed275f36-fb43-4825-aa59-5b3b0da5c62f (ed275f36-fb43-4825-aa59-5b3b0da5c62f): created successfully >2018-08-31 04:00:57.656 288514 INFO cinder.volume.manager [req-f6e0244c-b398-4fe3-87e5-18219ca2fe0b 64fc12ed066b494e95b98d115be08441 ea69b8db42db4aeb99c9fefc06dcca53 - default default] Created volume successfully. >2018-08-31 04:01:01.494 288514 INFO cinder.volume.manager [req-923f1ae0-5e98-4845-a3c2-c4df80bdcf24 64fc12ed066b494e95b98d115be08441 ea69b8db42db4aeb99c9fefc06dcca53 - default default] Deleted volume successfully. >2018-08-31 04:01:05.967 288514 INFO cinder.volume.flows.manager.create_volume [req-5a3ea54d-d743-481c-b628-084cc7aad9e2 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Volume 88fa03ff-69f8-4627-b129-32ce6577a608: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-88fa03ff-69f8-4627-b129-32ce6577a608', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f636d1a5cd0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 04:01:06.888 288514 INFO cinder.image.image_utils [req-5a3ea54d-d743-481c-b628-084cc7aad9e2 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-31 04:01:14.275 288514 INFO os_brick.initiator.connectors.iscsi [req-5a3ea54d-d743-481c-b628-084cc7aad9e2 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 04:01:22.153 288514 INFO cinder.image.image_utils [req-5a3ea54d-d743-481c-b628-084cc7aad9e2 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Converted 39.00 MB image at 36.55 MB/s >2018-08-31 04:01:26.567 288514 INFO cinder.volume.flows.manager.create_volume [req-5a3ea54d-d743-481c-b628-084cc7aad9e2 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Volume volume-88fa03ff-69f8-4627-b129-32ce6577a608 (88fa03ff-69f8-4627-b129-32ce6577a608): created successfully >2018-08-31 04:01:26.570 288514 INFO cinder.volume.manager [req-5a3ea54d-d743-481c-b628-084cc7aad9e2 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Created volume successfully. >2018-08-31 04:01:27.682 288514 INFO cinder.volume.flows.manager.create_volume [req-324cba11-27eb-4967-9bf4-7839b21f7c0d ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Volume 82ef9675-5d50-4e48-aa60-8b0293a50121: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-82ef9675-5d50-4e48-aa60-8b0293a50121', 'source_volid': '88fa03ff-69f8-4627-b129-32ce6577a608', 'source_volstatus': u'available'} >2018-08-31 04:01:30.045 288514 INFO cinder.volume.flows.manager.create_volume [req-324cba11-27eb-4967-9bf4-7839b21f7c0d ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Volume volume-82ef9675-5d50-4e48-aa60-8b0293a50121 (82ef9675-5d50-4e48-aa60-8b0293a50121): created successfully >2018-08-31 04:01:30.049 288514 INFO cinder.volume.manager [req-324cba11-27eb-4967-9bf4-7839b21f7c0d ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Created volume successfully. >2018-08-31 04:01:34.777 288514 INFO cinder.volume.manager [req-1eb08809-3a8a-4729-9280-a9f7e85ee890 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Deleted volume successfully. >2018-08-31 04:01:36.433 288514 INFO cinder.volume.manager [req-67a0ebc4-bf66-40e5-b959-5a493e720757 ce0d05f027f84dd3b73206e8853c123b e4dd436d41164c0b8d2c40d8db9dcd56 - default default] Deleted volume successfully. >2018-08-31 04:01:40.882 288514 INFO cinder.volume.flows.manager.create_volume [req-886f8763-3e4c-4f6c-af5f-6120980d22b0 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Volume 42642087-1bf1-421b-96ca-75abb54a46ff: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-42642087-1bf1-421b-96ca-75abb54a46ff'} >2018-08-31 04:01:42.180 288514 INFO cinder.volume.flows.manager.create_volume [req-886f8763-3e4c-4f6c-af5f-6120980d22b0 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Volume volume-42642087-1bf1-421b-96ca-75abb54a46ff (42642087-1bf1-421b-96ca-75abb54a46ff): created successfully >2018-08-31 04:01:42.184 288514 INFO cinder.volume.manager [req-886f8763-3e4c-4f6c-af5f-6120980d22b0 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Created volume successfully. >2018-08-31 04:01:43.287 288753 INFO cinder.volume.flows.manager.create_volume [req-11bc5f29-cf41-49a1-a948-91b56393dbbf d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Volume 7c6ea04f-41b3-461a-b00d-27feb21b1211: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7c6ea04f-41b3-461a-b00d-27feb21b1211'} >2018-08-31 04:01:44.646 288753 INFO cinder.volume.flows.manager.create_volume [req-11bc5f29-cf41-49a1-a948-91b56393dbbf d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Volume volume-7c6ea04f-41b3-461a-b00d-27feb21b1211 (7c6ea04f-41b3-461a-b00d-27feb21b1211): created successfully >2018-08-31 04:01:44.651 288753 INFO cinder.volume.manager [req-11bc5f29-cf41-49a1-a948-91b56393dbbf d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Created volume successfully. >2018-08-31 04:01:46.545 288514 INFO cinder.volume.flows.manager.create_volume [req-71d6723a-ef4d-45d4-a12f-888429cc6736 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Volume 463c6695-e0bf-4c10-9580-2cca2d62dd78: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-463c6695-e0bf-4c10-9580-2cca2d62dd78'} >2018-08-31 04:01:47.842 288514 INFO cinder.volume.flows.manager.create_volume [req-71d6723a-ef4d-45d4-a12f-888429cc6736 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Volume volume-463c6695-e0bf-4c10-9580-2cca2d62dd78 (463c6695-e0bf-4c10-9580-2cca2d62dd78): created successfully >2018-08-31 04:01:47.846 288514 INFO cinder.volume.manager [req-71d6723a-ef4d-45d4-a12f-888429cc6736 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Created volume successfully. >2018-08-31 04:01:53.749 288514 INFO cinder.volume.manager [req-52413adf-3b61-42eb-aea5-bbb6725953fd d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Deleted volume successfully. >2018-08-31 04:01:53.792 288753 INFO cinder.volume.manager [req-88d19fab-edcd-4e7b-be49-0ac0b0679ecb d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Deleted volume successfully. >2018-08-31 04:01:54.058 288514 INFO cinder.volume.manager [req-b8fd745b-4f72-428d-87eb-e8695a6d3df5 d4b80d583c394ae19bcbe08ec4c5dd34 566b3cb18f5349c781b8331ad61b185c - default default] Deleted volume successfully. >2018-08-31 04:02:14.523 288514 INFO cinder.volume.flows.manager.create_volume [req-6dc2d367-a31a-49c5-a6f8-fdc3a87ad5ca d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Volume 974fc191-64e3-4fdd-97d7-73a994683218: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-974fc191-64e3-4fdd-97d7-73a994683218'} >2018-08-31 04:02:15.845 288514 INFO cinder.volume.flows.manager.create_volume [req-6dc2d367-a31a-49c5-a6f8-fdc3a87ad5ca d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Volume volume-974fc191-64e3-4fdd-97d7-73a994683218 (974fc191-64e3-4fdd-97d7-73a994683218): created successfully >2018-08-31 04:02:15.848 288514 INFO cinder.volume.manager [req-6dc2d367-a31a-49c5-a6f8-fdc3a87ad5ca d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Created volume successfully. >2018-08-31 04:02:16.537 288514 INFO cinder.rpc [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Automatically selected cinder-volume objects version 1.27 as minimum service version. >2018-08-31 04:02:16.542 288514 INFO cinder.rpc [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Automatically selected cinder-volume RPC version 3.15 as minimum service version. >2018-08-31 04:02:16.699 288753 INFO cinder.volume.flows.manager.create_volume [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Volume ca68034a-6871-49b0-b8cd-60fa12db5361: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ca68034a-6871-49b0-b8cd-60fa12db5361'} >2018-08-31 04:02:18.068 288753 INFO cinder.volume.flows.manager.create_volume [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Volume volume-ca68034a-6871-49b0-b8cd-60fa12db5361 (ca68034a-6871-49b0-b8cd-60fa12db5361): created successfully >2018-08-31 04:02:18.072 288753 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Created volume successfully. >2018-08-31 04:02:27.665 288753 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Initialize volume connection completed successfully. >2018-08-31 04:02:27.672 288514 INFO os_brick.initiator.connectors.iscsi [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Trying to connect to iSCSI portal 9.11.54.170:3260 >2018-08-31 04:02:34.045 288514 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Initialize volume connection completed successfully. >2018-08-31 04:02:34.045 288514 INFO os_brick.initiator.connectors.iscsi [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 04:02:59.635 288514 INFO cinder.volume.utils [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Volume copy 1024.00 MB at 51.35 MB/s >2018-08-31 04:03:04.202 288753 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Terminate volume connection completed successfully. >2018-08-31 04:03:04.262 288753 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Remove volume export completed successfully. >2018-08-31 04:03:09.028 288514 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Terminate volume connection completed successfully. >2018-08-31 04:03:09.064 288514 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Remove volume export completed successfully. >2018-08-31 04:03:09.313 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 04:03:09.785 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Error running SSH command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 04:03:09.787 288753 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 04:03:09.788 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Error running SSH command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 04:03:09.789 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-974fc191-64e3-4fdd-97d7-73a994683218', '"volume-ca68034a-6871-49b0-b8cd-60fa12db5361"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-974fc191-64e3-4fdd-97d7-73a994683218 "volume-ca68034a-6871-49b0-b8cd-60fa12db5361" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 04:03:09.790 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Unable to rename the logical volume for volume: 974fc191-64e3-4fdd-97d7-73a994683218: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-974fc191-64e3-4fdd-97d7-73a994683218', '"volume-ca68034a-6871-49b0-b8cd-60fa12db5361"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-31 04:03:09.956 288514 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Complete-Migrate volume completed successfully. >2018-08-31 04:03:09.957 288514 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Migrate volume completed successfully. >2018-08-31 04:03:10.548 288514 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Retype volume completed successfully. >2018-08-31 04:03:12.110 288514 INFO cinder.volume.manager [req-68094e8a-f727-451d-a171-9b7ba5f5aba0 d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Deleted volume successfully. >2018-08-31 04:03:15.049 288753 WARNING cinder.quota [req-7eecb585-9d0a-4fd4-a825-afa81eea999c d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1327234034 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1327234034, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 04:03:15.051 288753 WARNING cinder.quota [req-7eecb585-9d0a-4fd4-a825-afa81eea999c d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1327234034 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1327234034, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 04:03:15.307 288753 INFO cinder.volume.manager [req-7eecb585-9d0a-4fd4-a825-afa81eea999c d2e6361df5eb42c2af30d67c635e11a7 1db3e342e17f4adc9c87762a01e444c1 - default default] Deleted volume successfully. >2018-08-31 04:03:36.172 288753 INFO cinder.volume.flows.manager.create_volume [req-35f624fb-dd17-441b-ac82-da62bd969143 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Volume f993a26a-88a2-4bb2-a277-d8893fad545c: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f993a26a-88a2-4bb2-a277-d8893fad545c'} >2018-08-31 04:03:37.781 288753 INFO cinder.volume.flows.manager.create_volume [req-35f624fb-dd17-441b-ac82-da62bd969143 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Volume volume-f993a26a-88a2-4bb2-a277-d8893fad545c (f993a26a-88a2-4bb2-a277-d8893fad545c): created successfully >2018-08-31 04:03:37.785 288753 INFO cinder.volume.manager [req-35f624fb-dd17-441b-ac82-da62bd969143 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Created volume successfully. >2018-08-31 04:03:42.231 288753 INFO cinder.volume.manager [req-6002784b-38c7-4717-975e-e05af0415728 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Create snapshot completed successfully >2018-08-31 04:03:45.822 288753 INFO cinder.volume.manager [req-f8c5a2b3-3523-46fa-b485-cb995e0d9a56 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Create snapshot completed successfully >2018-08-31 04:03:50.260 288753 INFO cinder.volume.manager [req-90e1955e-44d3-4c2b-bfeb-291945af4bb3 84b3f2b1b59745519efaafdfd0de38ae c262066a88274e728f7abf06f9f868b0 - default default] Delete snapshot completed successfully. >2018-08-31 04:03:54.058 288753 INFO cinder.volume.manager [req-068ab7cf-f616-44cc-be18-04f3635e6cd3 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Create snapshot completed successfully >2018-08-31 04:03:58.726 288753 INFO cinder.volume.manager [req-981415ad-43d9-40ef-bf24-0025278213c6 84b3f2b1b59745519efaafdfd0de38ae c262066a88274e728f7abf06f9f868b0 - default default] Delete snapshot completed successfully. >2018-08-31 04:04:03.365 288753 INFO cinder.volume.manager [req-204a5033-dfa0-4156-9f3b-b73bcea17d81 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Create snapshot completed successfully >2018-08-31 04:04:07.430 288753 INFO cinder.volume.manager [req-34c0e2d6-51cb-4a6f-9773-234825439fbf 84b3f2b1b59745519efaafdfd0de38ae c262066a88274e728f7abf06f9f868b0 - default default] Delete snapshot completed successfully. >2018-08-31 04:04:11.450 288753 INFO cinder.volume.manager [req-ba8967bf-7dd6-4bc1-9eaa-8f5f674e05f7 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Create snapshot completed successfully >2018-08-31 04:04:16.055 288753 INFO cinder.volume.manager [req-1cc1f17f-73a7-482d-a665-61b9e16f4f6a 84b3f2b1b59745519efaafdfd0de38ae c262066a88274e728f7abf06f9f868b0 - default default] Delete snapshot completed successfully. >2018-08-31 04:04:21.113 288753 INFO cinder.volume.manager [req-548c78b5-d22d-4e23-a1e6-c9e94c881e9f 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Delete snapshot completed successfully. >2018-08-31 04:04:23.647 288753 INFO cinder.volume.manager [req-47c285f4-0215-4fa4-9ffe-36518c7ea271 555a3554a90c4a609a85b3e63704a9d5 19adfb9d099f4c3b9701301dc869974d - default default] Deleted volume successfully. >2018-08-31 04:04:27.507 288753 INFO cinder.volume.flows.manager.create_volume [req-6b263323-d6de-4c9b-8193-df9ab4bc6337 d9098bec7e8d4ef9a5940fff7b7b9039 92f2d7f4f2eb48c59e9b2f7cde900045 - default default] Volume 9edac985-6fda-481a-95c8-3c8013199c17: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9edac985-6fda-481a-95c8-3c8013199c17'} >2018-08-31 04:04:28.916 288753 INFO cinder.volume.flows.manager.create_volume [req-6b263323-d6de-4c9b-8193-df9ab4bc6337 d9098bec7e8d4ef9a5940fff7b7b9039 92f2d7f4f2eb48c59e9b2f7cde900045 - default default] Volume volume-9edac985-6fda-481a-95c8-3c8013199c17 (9edac985-6fda-481a-95c8-3c8013199c17): created successfully >2018-08-31 04:04:28.920 288753 INFO cinder.volume.manager [req-6b263323-d6de-4c9b-8193-df9ab4bc6337 d9098bec7e8d4ef9a5940fff7b7b9039 92f2d7f4f2eb48c59e9b2f7cde900045 - default default] Created volume successfully. >2018-08-31 04:04:33.376 288753 INFO cinder.volume.manager [req-230194bd-7544-447a-8588-0604c0c01bfd d9098bec7e8d4ef9a5940fff7b7b9039 92f2d7f4f2eb48c59e9b2f7cde900045 - default default] Create snapshot completed successfully >2018-08-31 04:04:39.074 288753 INFO cinder.volume.manager [req-4e7fcb8f-cd4f-41ca-b445-10310db10729 d9098bec7e8d4ef9a5940fff7b7b9039 92f2d7f4f2eb48c59e9b2f7cde900045 - default default] Delete snapshot completed successfully. >2018-08-31 04:04:41.409 288753 INFO cinder.volume.manager [req-f89ce02c-8896-49c8-b195-5106e829e34d d9098bec7e8d4ef9a5940fff7b7b9039 92f2d7f4f2eb48c59e9b2f7cde900045 - default default] Deleted volume successfully. >2018-08-31 04:04:44.859 288753 INFO cinder.volume.flows.manager.create_volume [req-77bd1b20-6ac7-40d9-804b-707faf08f21d 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Volume be6648a9-7f20-48fd-add6-017782dfc0d5: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-be6648a9-7f20-48fd-add6-017782dfc0d5'} >2018-08-31 04:04:46.351 288753 INFO cinder.volume.flows.manager.create_volume [req-77bd1b20-6ac7-40d9-804b-707faf08f21d 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Volume volume-be6648a9-7f20-48fd-add6-017782dfc0d5 (be6648a9-7f20-48fd-add6-017782dfc0d5): created successfully >2018-08-31 04:04:46.355 288753 INFO cinder.volume.manager [req-77bd1b20-6ac7-40d9-804b-707faf08f21d 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Created volume successfully. >2018-08-31 04:04:50.904 288753 INFO cinder.volume.manager [req-684719d2-2d39-4f49-9025-9ac6579ccd48 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:05:02.334 288753 INFO cinder.volume.manager [req-4fba4c9f-b79a-4925-9319-13a548de250b 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Initialize volume connection completed successfully. >2018-08-31 04:05:34.458 288753 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 04:05:34.555 288753 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 04:05:35.915 288753 INFO cinder.volume.manager [req-4fba4c9f-b79a-4925-9319-13a548de250b 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Terminate volume connection completed successfully. >2018-08-31 04:05:35.957 288753 INFO cinder.volume.manager [req-4fba4c9f-b79a-4925-9319-13a548de250b 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Remove volume export completed successfully. >2018-08-31 04:05:37.963 288753 INFO cinder.volume.manager [req-4fba4c9f-b79a-4925-9319-13a548de250b 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Deleted volume successfully. >2018-08-31 04:05:52.698 288753 INFO cinder.volume.manager [req-43621bfb-eb81-45e1-b915-6744efa1e374 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Initialize volume connection completed successfully. >2018-08-31 04:06:02.424 288753 INFO cinder.volume.manager [req-723f4750-1110-40b0-987a-8578225a6a79 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Attaching volume be6648a9-7f20-48fd-add6-017782dfc0d5 to instance cfab8d93-cab7-445b-83df-81e8432fb858 at mountpoint /dev/vdb on host None. >2018-08-31 04:06:02.513 288753 INFO cinder.volume.manager [req-723f4750-1110-40b0-987a-8578225a6a79 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Attach volume completed successfully. >2018-08-31 04:06:07.089 288753 INFO cinder.volume.manager [req-1625bda7-d931-409c-a6b1-393e6485a81b 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:06:10.244 288753 INFO cinder.volume.manager [req-0d936d97-e88d-44eb-a415-6df22362619f 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:06:13.687 288753 INFO cinder.volume.manager [req-fd30a1a7-18cf-447f-9655-6fa04c20c713 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:06:16.590 288753 INFO cinder.volume.manager [req-98b30e31-cb21-4eb3-9611-1241b9ca477a 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:06:19.817 288753 INFO cinder.volume.manager [req-348d4ce1-0c07-4bb8-902a-c55284914041 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:06:23.056 288753 INFO cinder.volume.manager [req-366fa9fd-67ec-4690-95b4-56c4a8eb8a80 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:06:33.539 288753 INFO cinder.volume.manager [req-6aceec09-006a-4e80-b661-29d3ec99b381 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Terminate volume connection completed successfully. >2018-08-31 04:06:33.693 288753 INFO cinder.volume.manager [req-4b1cde26-b05d-4f67-942e-761c81936e29 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Detaching volume be6648a9-7f20-48fd-add6-017782dfc0d5 from instance cfab8d93-cab7-445b-83df-81e8432fb858. >2018-08-31 04:06:33.780 288753 INFO cinder.volume.manager [req-4b1cde26-b05d-4f67-942e-761c81936e29 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Detach volume completed successfully. >2018-08-31 04:06:38.447 288753 INFO cinder.volume.manager [req-038d8d70-ef77-4ab2-be12-c2425fcb7ca7 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:06:42.262 288753 INFO cinder.volume.manager [req-a54947c1-cca7-4f76-bc38-9232f5539ecb 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:06:45.707 288753 INFO cinder.volume.manager [req-78399332-eddc-4f29-a7a4-faeb566cc390 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:07:01.101 288753 INFO cinder.volume.manager [req-a1dc6d34-d8d1-4dbd-8f4e-84f9b5a14c48 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Initialize volume connection completed successfully. >2018-08-31 04:07:10.324 288753 INFO cinder.volume.manager [req-5ad9cabb-c843-4c17-b498-cea69a12ef96 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Attaching volume be6648a9-7f20-48fd-add6-017782dfc0d5 to instance acd51114-5639-4a22-b9fe-43a3a1323651 at mountpoint /dev/vdb on host None. >2018-08-31 04:07:10.396 288753 INFO cinder.volume.manager [req-5ad9cabb-c843-4c17-b498-cea69a12ef96 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Attach volume completed successfully. >2018-08-31 04:07:13.384 288753 INFO cinder.volume.manager [req-ea1e9cef-b1dd-4312-985d-b06472fc3715 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:07:16.686 288753 INFO cinder.volume.manager [req-5802153c-7d7f-4192-b675-061e1d623286 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:07:20.293 288753 INFO cinder.volume.manager [req-1caeaf98-9698-4065-a1f1-0fa228e2478b 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:07:23.687 288753 INFO cinder.volume.manager [req-911cd841-04b0-4b27-9a6a-c56b2c28ec35 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:07:26.732 288753 INFO cinder.volume.manager [req-e002b0e6-c620-4fac-a2e8-ac3613d230c5 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:07:36.920 288753 INFO cinder.volume.manager [req-d65bed05-cf8d-44ba-b9f4-711324d1a333 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Terminate volume connection completed successfully. >2018-08-31 04:07:37.067 288753 INFO cinder.volume.manager [req-e4d27236-6cb2-4009-852c-37ae06bec110 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Detaching volume be6648a9-7f20-48fd-add6-017782dfc0d5 from instance acd51114-5639-4a22-b9fe-43a3a1323651. >2018-08-31 04:07:37.155 288753 INFO cinder.volume.manager [req-e4d27236-6cb2-4009-852c-37ae06bec110 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Detach volume completed successfully. >2018-08-31 04:07:39.839 288753 INFO cinder.volume.flows.manager.create_volume [req-2e647549-e7c6-4cfa-85ff-8c2fffdf65e7 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Volume 24eeaced-a98d-4d87-b492-6f035ef8a3b2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-24eeaced-a98d-4d87-b492-6f035ef8a3b2'} >2018-08-31 04:07:41.242 288753 INFO cinder.volume.flows.manager.create_volume [req-2e647549-e7c6-4cfa-85ff-8c2fffdf65e7 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Volume volume-24eeaced-a98d-4d87-b492-6f035ef8a3b2 (24eeaced-a98d-4d87-b492-6f035ef8a3b2): created successfully >2018-08-31 04:07:41.246 288753 INFO cinder.volume.manager [req-2e647549-e7c6-4cfa-85ff-8c2fffdf65e7 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Created volume successfully. >2018-08-31 04:07:44.983 288753 INFO cinder.volume.manager [req-0929faf5-4a3d-458d-af31-25e0361fae4d 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Create snapshot completed successfully >2018-08-31 04:07:45.661 288753 INFO cinder.volume.flows.manager.create_volume [req-9031c47a-1ba4-454f-807d-275d7059e22c 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Volume a3e850f5-d719-4082-a589-8a80b094e240: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': 'f4eb738c-1032-49ce-aed7-97c2f5b1faee', 'volume_name': 'volume-a3e850f5-d719-4082-a589-8a80b094e240'} >2018-08-31 04:08:01.031 288753 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-9031c47a-1ba4-454f-807d-275d7059e22c 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Unable to get remote copy information for volume volume-a3e850f5-d719-4082-a589-8a80b094e240 >2018-08-31 04:08:02.237 288753 INFO cinder.volume.flows.manager.create_volume [req-9031c47a-1ba4-454f-807d-275d7059e22c 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Volume volume-a3e850f5-d719-4082-a589-8a80b094e240 (a3e850f5-d719-4082-a589-8a80b094e240): created successfully >2018-08-31 04:08:02.241 288753 INFO cinder.volume.manager [req-9031c47a-1ba4-454f-807d-275d7059e22c 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Created volume successfully. >2018-08-31 04:08:05.066 288753 INFO cinder.volume.manager [req-0e76f3c7-7d5d-4bd3-91a9-6a42fd3345e4 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Deleted volume successfully. >2018-08-31 04:08:10.132 288753 INFO cinder.volume.manager [req-ce255cd2-42fe-47e1-9020-1a81e788a24c 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:08:11.145 288753 INFO cinder.volume.manager [req-0b84c823-7d9b-4c8f-91c4-7197604bbd8c 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Delete snapshot completed successfully. >2018-08-31 04:08:15.396 288753 WARNING cinder.db.sqlalchemy.api [req-c3b2bca9-e4d9-452f-a32c-3c616cade33e 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-31 04:08:15.464 288753 INFO cinder.volume.manager [req-0473f364-f9ba-4317-a49d-0e00582b7f5f 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Deleted volume successfully. >2018-08-31 04:08:15.768 288753 INFO cinder.volume.manager [req-c3b2bca9-e4d9-452f-a32c-3c616cade33e 4da3c27180b34168a6ca1b01c9fc18fb eccc6fc51f8c4503951f99539a2bf22f - default default] Deleted volume successfully. >2018-08-31 04:08:19.203 288753 INFO cinder.volume.flows.manager.create_volume [req-f0460efc-e4dd-441d-a7ce-8885718d59a2 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Volume 95e59a51-f67e-4760-ba02-fc1e57ab3116: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-95e59a51-f67e-4760-ba02-fc1e57ab3116'} >2018-08-31 04:08:20.492 288753 INFO cinder.volume.flows.manager.create_volume [req-f0460efc-e4dd-441d-a7ce-8885718d59a2 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Volume volume-95e59a51-f67e-4760-ba02-fc1e57ab3116 (95e59a51-f67e-4760-ba02-fc1e57ab3116): created successfully >2018-08-31 04:08:20.496 288753 INFO cinder.volume.manager [req-f0460efc-e4dd-441d-a7ce-8885718d59a2 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Created volume successfully. >2018-08-31 04:08:23.605 288753 INFO cinder.volume.manager [req-d072ab09-6798-44ef-bd95-944ab03553cc 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Create snapshot completed successfully >2018-08-31 04:08:27.458 288753 INFO cinder.volume.manager [req-99daf2ef-c79a-4bc5-9787-b584ce713e46 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Create snapshot completed successfully >2018-08-31 04:08:31.224 288753 INFO cinder.volume.manager [req-329a3e44-c12e-488b-9464-8478e1471b1b 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Create snapshot completed successfully >2018-08-31 04:08:33.972 288753 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 04:08:34.059 288753 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 04:08:34.384 288753 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 04:08:34.471 288753 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 04:08:39.586 288753 INFO cinder.volume.manager [req-2d4aa4dd-9374-4d07-8191-e3fde02314c0 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Delete snapshot completed successfully. >2018-08-31 04:08:40.659 288753 INFO cinder.volume.manager [req-4e5f6e99-42cb-43c4-bbaf-f3d89c196a2d 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Delete snapshot completed successfully. >2018-08-31 04:08:41.682 288753 INFO cinder.volume.manager [req-cfc108d7-f2f0-45aa-a084-c5c23703dc39 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Delete snapshot completed successfully. >2018-08-31 04:08:44.739 288753 INFO cinder.volume.manager [req-4158f1ce-0814-43ec-bf55-6e63ff7efe93 174cb4402ba54134a2c7958a81336c66 ed8a4f8cf2d04b2d9790233959567fd3 - default default] Deleted volume successfully. >2018-08-31 04:08:49.442 288753 INFO cinder.volume.flows.manager.create_volume [req-142567cc-4d7a-4674-a2b4-ee9f627de2ad bbdec099a04f45109c6895a1f74f8f19 7401801d3e4d45f7ad80b3e383c88a58 - default default] Volume 39066aaf-2f6e-49aa-b9a2-1d26d3cbedd9: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-39066aaf-2f6e-49aa-b9a2-1d26d3cbedd9'} >2018-08-31 04:08:50.841 288753 INFO cinder.volume.flows.manager.create_volume [req-142567cc-4d7a-4674-a2b4-ee9f627de2ad bbdec099a04f45109c6895a1f74f8f19 7401801d3e4d45f7ad80b3e383c88a58 - default default] Volume volume-39066aaf-2f6e-49aa-b9a2-1d26d3cbedd9 (39066aaf-2f6e-49aa-b9a2-1d26d3cbedd9): created successfully >2018-08-31 04:08:50.845 288753 INFO cinder.volume.manager [req-142567cc-4d7a-4674-a2b4-ee9f627de2ad bbdec099a04f45109c6895a1f74f8f19 7401801d3e4d45f7ad80b3e383c88a58 - default default] Created volume successfully. >2018-08-31 04:08:54.545 288753 INFO cinder.volume.manager [req-06a6370f-0949-4874-8daf-e0b323d9a6f8 bbdec099a04f45109c6895a1f74f8f19 7401801d3e4d45f7ad80b3e383c88a58 - default default] Create snapshot completed successfully >2018-08-31 04:08:58.730 288753 INFO cinder.volume.manager [req-6a1e2b54-eb0c-49ef-92f0-126ec904c7e5 bbdec099a04f45109c6895a1f74f8f19 7401801d3e4d45f7ad80b3e383c88a58 - default default] Delete snapshot completed successfully. >2018-08-31 04:09:01.782 288753 INFO cinder.volume.manager [req-d5f2eeee-5ed9-4713-bfb3-a21eb159f1ba bbdec099a04f45109c6895a1f74f8f19 7401801d3e4d45f7ad80b3e383c88a58 - default default] Deleted volume successfully. >2018-08-31 04:09:24.958 288753 WARNING oslo_config.cfg [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens). Its value may be silently ignored in the future. >2018-08-31 04:09:25.285 288753 INFO cinder.volume.flows.manager.create_volume [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume ca3d3b43-9f08-4e3e-bd77-b04693ff135d: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-ca3d3b43-9f08-4e3e-bd77-b04693ff135d', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f636cfd1b50>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 04:09:26.396 288753 INFO cinder.image.image_utils [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Image download 12.00 MB at 10.83 MB/s >2018-08-31 04:09:28.686 288753 INFO oslo.privsep.daemon [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmp0LnS8f/privsep.sock'] >2018-08-31 04:09:29.460 288753 WARNING oslo.privsep.daemon [-] privsep log: Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT". >2018-08-31 04:09:29.555 288753 INFO oslo.privsep.daemon [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Spawned new privsep daemon via rootwrap >2018-08-31 04:09:29.502 467881 INFO oslo.privsep.daemon [-] privsep daemon starting >2018-08-31 04:09:29.505 467881 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 >2018-08-31 04:09:29.508 467881 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none >2018-08-31 04:09:29.508 467881 INFO oslo.privsep.daemon [-] privsep daemon running as pid 467881 >2018-08-31 04:09:34.031 288753 INFO os_brick.initiator.connectors.iscsi [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 04:09:41.850 288753 INFO cinder.image.image_utils [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Converted 39.00 MB image at 38.10 MB/s >2018-08-31 04:09:45.702 288753 INFO cinder.volume.flows.manager.create_volume [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume volume-ca3d3b43-9f08-4e3e-bd77-b04693ff135d (ca3d3b43-9f08-4e3e-bd77-b04693ff135d): created successfully >2018-08-31 04:09:45.707 288753 INFO cinder.volume.manager [req-5e67cfc4-eb90-40d2-a966-be3d63949f19 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Created volume successfully. >2018-08-31 04:09:47.301 288753 INFO cinder.volume.flows.manager.create_volume [req-7b3e443a-dcec-407b-abf2-d6bfe3ccc60a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume b88ee42b-0cec-40a1-8ed9-4642527873be: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b88ee42b-0cec-40a1-8ed9-4642527873be', 'source_volid': 'ca3d3b43-9f08-4e3e-bd77-b04693ff135d', 'source_volstatus': u'available'} >2018-08-31 04:09:49.610 288753 INFO cinder.volume.flows.manager.create_volume [req-7b3e443a-dcec-407b-abf2-d6bfe3ccc60a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume volume-b88ee42b-0cec-40a1-8ed9-4642527873be (b88ee42b-0cec-40a1-8ed9-4642527873be): created successfully >2018-08-31 04:09:49.615 288753 INFO cinder.volume.manager [req-7b3e443a-dcec-407b-abf2-d6bfe3ccc60a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Created volume successfully. >2018-08-31 04:09:50.852 288753 INFO cinder.volume.flows.manager.create_volume [req-4593379f-f80a-4434-b7a1-ae123336c111 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume 91841bdc-02c2-49e6-ba9e-ccb8886d9602: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-91841bdc-02c2-49e6-ba9e-ccb8886d9602'} >2018-08-31 04:09:52.141 288753 INFO cinder.volume.flows.manager.create_volume [req-4593379f-f80a-4434-b7a1-ae123336c111 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume volume-91841bdc-02c2-49e6-ba9e-ccb8886d9602 (91841bdc-02c2-49e6-ba9e-ccb8886d9602): created successfully >2018-08-31 04:09:52.144 288753 INFO cinder.volume.manager [req-4593379f-f80a-4434-b7a1-ae123336c111 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Created volume successfully. >2018-08-31 04:09:53.333 288753 INFO cinder.volume.flows.manager.create_volume [req-e9740238-cff0-44a8-ae5f-0d5d6a39c04a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume c7bae57d-dd8e-4145-a675-071066cb0563: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-c7bae57d-dd8e-4145-a675-071066cb0563', 'source_volid': '91841bdc-02c2-49e6-ba9e-ccb8886d9602', 'source_volstatus': u'available'} >2018-08-31 04:10:02.685 288753 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-e9740238-cff0-44a8-ae5f-0d5d6a39c04a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Unable to get remote copy information for volume volume-c7bae57d-dd8e-4145-a675-071066cb0563 >2018-08-31 04:10:03.933 288753 INFO cinder.volume.flows.manager.create_volume [req-e9740238-cff0-44a8-ae5f-0d5d6a39c04a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Volume volume-c7bae57d-dd8e-4145-a675-071066cb0563 (c7bae57d-dd8e-4145-a675-071066cb0563): created successfully >2018-08-31 04:10:03.937 288753 INFO cinder.volume.manager [req-e9740238-cff0-44a8-ae5f-0d5d6a39c04a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Created volume successfully. >2018-08-31 04:10:09.575 288753 INFO cinder.volume.manager [req-54a1da54-aae1-483d-9ef3-648897d60c3e 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Deleted volume successfully. >2018-08-31 04:10:09.980 288753 WARNING oslo.service.loopingcall [-] Function 'cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common.StorwizeHelpers._check_vdisk_fc_mappings' run outlasted interval by 1.73 sec >2018-08-31 04:10:09.985 288753 INFO cinder.volume.manager [req-f87faacd-401f-4fbf-9c4d-19375a2f556a 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Deleted volume successfully. >2018-08-31 04:10:12.847 288753 INFO cinder.volume.manager [req-213bc196-8c05-408b-ae21-157dff937671 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Deleted volume successfully. >2018-08-31 04:10:13.107 288753 INFO cinder.volume.manager [req-2ca493cd-ce4c-4347-a8d9-193f3455ef01 896e04954bf54bd4b765bbdba2fd1f35 13245cfc96694040859941f09543d158 - default default] Deleted volume successfully. >2018-08-31 04:10:17.003 288514 INFO cinder.volume.flows.manager.create_volume [req-125f8617-c8fc-4630-9ff4-9112e993f864 e66ab2d95df8435b9c32b2ba5d8e8f8a 40bfac7873c64ef6b8d19fe61dd5ee3b - default default] Volume d48427cb-2bd9-4625-b365-b67a7f5a49c7: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-d48427cb-2bd9-4625-b365-b67a7f5a49c7'} >2018-08-31 04:10:18.461 288514 INFO cinder.volume.flows.manager.create_volume [req-125f8617-c8fc-4630-9ff4-9112e993f864 e66ab2d95df8435b9c32b2ba5d8e8f8a 40bfac7873c64ef6b8d19fe61dd5ee3b - default default] Volume volume-d48427cb-2bd9-4625-b365-b67a7f5a49c7 (d48427cb-2bd9-4625-b365-b67a7f5a49c7): created successfully >2018-08-31 04:10:18.465 288514 INFO cinder.volume.manager [req-125f8617-c8fc-4630-9ff4-9112e993f864 e66ab2d95df8435b9c32b2ba5d8e8f8a 40bfac7873c64ef6b8d19fe61dd5ee3b - default default] Created volume successfully. >2018-08-31 04:10:22.038 288514 INFO cinder.volume.manager [req-77dec4ae-6e68-4f1a-b283-46c97b4e214f e66ab2d95df8435b9c32b2ba5d8e8f8a 40bfac7873c64ef6b8d19fe61dd5ee3b - default default] Deleted volume successfully. >2018-08-31 04:10:47.095 288514 INFO cinder.volume.flows.manager.create_volume [req-4ac41457-2d7c-4525-b0c3-e9a0fbcb03c3 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] Volume 96e5ecdc-c234-4abd-bc98-362b3f671959: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-96e5ecdc-c234-4abd-bc98-362b3f671959'} >2018-08-31 04:10:48.425 288514 INFO cinder.volume.flows.manager.create_volume [req-4ac41457-2d7c-4525-b0c3-e9a0fbcb03c3 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] Volume volume-96e5ecdc-c234-4abd-bc98-362b3f671959 (96e5ecdc-c234-4abd-bc98-362b3f671959): created successfully >2018-08-31 04:10:48.429 288514 INFO cinder.volume.manager [req-4ac41457-2d7c-4525-b0c3-e9a0fbcb03c3 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] Created volume successfully. >2018-08-31 04:10:50.881 288514 INFO cinder.volume.manager [req-e7a8c769-d815-4867-8c7a-5527341466d1 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] Deleted volume successfully. >2018-08-31 04:10:52.175 288514 INFO cinder.volume.flows.manager.create_volume [req-ae56a0f8-3184-4b24-8499-b6e8bf83b4e9 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] Volume f3d129a0-ba37-44b8-b938-691cbbb758c4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f3d129a0-ba37-44b8-b938-691cbbb758c4'} >2018-08-31 04:10:53.461 288514 INFO cinder.volume.flows.manager.create_volume [req-ae56a0f8-3184-4b24-8499-b6e8bf83b4e9 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] Volume volume-f3d129a0-ba37-44b8-b938-691cbbb758c4 (f3d129a0-ba37-44b8-b938-691cbbb758c4): created successfully >2018-08-31 04:10:53.465 288514 INFO cinder.volume.manager [req-ae56a0f8-3184-4b24-8499-b6e8bf83b4e9 482d530ccd3f4e7e9fc51b1092f66e0f ef8a3a7f4bd945569152c93f4feeda4b - default default] Created volume successfully. >2018-08-31 04:10:54.978 288514 INFO cinder.volume.manager [req-79410536-089d-4857-9982-5179fcc9792f 5b35a6b8cef54ae29306dfef850715bd 71dd02618df246489da73f5719b273ea - default default] Transfer volume completed successfully. >2018-08-31 04:10:58.293 288514 INFO cinder.volume.manager [req-6db6198b-2292-4218-aced-356d95657ee9 79f4ec99ee3c427bab73686add0dc8e4 70f0768e50fe413eaa32bb3d07956230 - default default] Deleted volume successfully. >2018-08-31 04:11:07.494 288514 INFO cinder.volume.flows.manager.create_volume [req-06d78353-80fd-463a-8254-e19c0045ec65 d9eb7d33c3564cabad025699899d6a48 b632f4599bc24d3baf50fa2145b63090 - default default] Volume 9a5266e9-a8bc-47ac-a324-aa961c7e4016: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9a5266e9-a8bc-47ac-a324-aa961c7e4016'} >2018-08-31 04:11:08.829 288514 INFO cinder.volume.flows.manager.create_volume [req-06d78353-80fd-463a-8254-e19c0045ec65 d9eb7d33c3564cabad025699899d6a48 b632f4599bc24d3baf50fa2145b63090 - default default] Volume volume-9a5266e9-a8bc-47ac-a324-aa961c7e4016 (9a5266e9-a8bc-47ac-a324-aa961c7e4016): created successfully >2018-08-31 04:11:08.833 288514 INFO cinder.volume.manager [req-06d78353-80fd-463a-8254-e19c0045ec65 d9eb7d33c3564cabad025699899d6a48 b632f4599bc24d3baf50fa2145b63090 - default default] Created volume successfully. >2018-08-31 04:11:12.675 288514 INFO cinder.volume.manager [req-2f1190ff-6730-440e-8801-225df1f1cf58 d9eb7d33c3564cabad025699899d6a48 b632f4599bc24d3baf50fa2145b63090 - default default] Deleted volume successfully. >2018-08-31 04:11:18.961 288514 INFO cinder.volume.flows.manager.create_volume [req-59cf0f4f-7df8-4bea-801e-23e8d131ded2 d3fa60f8ae7e442d830f12751b85fc61 69942bbd51bd476da4cca41b4dc6cdd7 - default default] Volume cdf04c23-8f52-435d-beb9-e64138a99803: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-cdf04c23-8f52-435d-beb9-e64138a99803'} >2018-08-31 04:11:20.242 288514 INFO cinder.volume.flows.manager.create_volume [req-59cf0f4f-7df8-4bea-801e-23e8d131ded2 d3fa60f8ae7e442d830f12751b85fc61 69942bbd51bd476da4cca41b4dc6cdd7 - default default] Volume volume-cdf04c23-8f52-435d-beb9-e64138a99803 (cdf04c23-8f52-435d-beb9-e64138a99803): created successfully >2018-08-31 04:11:20.246 288514 INFO cinder.volume.manager [req-59cf0f4f-7df8-4bea-801e-23e8d131ded2 d3fa60f8ae7e442d830f12751b85fc61 69942bbd51bd476da4cca41b4dc6cdd7 - default default] Created volume successfully. >2018-08-31 04:11:23.331 288514 INFO cinder.volume.manager [req-77384b67-02cd-4f99-aad7-b4cf7fcfe014 d3fa60f8ae7e442d830f12751b85fc61 69942bbd51bd476da4cca41b4dc6cdd7 - default default] Create snapshot completed successfully >2018-08-31 04:11:27.962 288514 INFO cinder.volume.manager [req-bf8a8912-eee4-454f-971c-84b8183f6dcb d3fa60f8ae7e442d830f12751b85fc61 69942bbd51bd476da4cca41b4dc6cdd7 - default default] Delete snapshot completed successfully. >2018-08-31 04:11:30.758 288514 INFO cinder.volume.manager [req-d53f3d35-dd12-4a5e-aa47-22fb45dad055 d3fa60f8ae7e442d830f12751b85fc61 69942bbd51bd476da4cca41b4dc6cdd7 - default default] Deleted volume successfully. >2018-08-31 04:49:24.459 288514 INFO cinder.volume.flows.manager.create_volume [req-e710c5d1-e225-4dc8-885d-00b2417b57fc 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Volume 5601ca28-6c8e-4411-ab14-a96cc9daf3bf: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5601ca28-6c8e-4411-ab14-a96cc9daf3bf'} >2018-08-31 04:49:25.759 288514 INFO cinder.volume.flows.manager.create_volume [req-e710c5d1-e225-4dc8-885d-00b2417b57fc 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Volume volume-5601ca28-6c8e-4411-ab14-a96cc9daf3bf (5601ca28-6c8e-4411-ab14-a96cc9daf3bf): created successfully >2018-08-31 04:49:25.763 288514 INFO cinder.volume.manager [req-e710c5d1-e225-4dc8-885d-00b2417b57fc 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Created volume successfully. >2018-08-31 04:49:43.748 288514 INFO cinder.volume.manager [req-3ee1390d-9ae3-40b9-a149-2ee886945c60 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Initialize volume connection completed successfully. >2018-08-31 04:49:52.233 288514 INFO cinder.volume.manager [req-d094220d-fb23-4dd8-b6c5-3ebd0b1fcfc8 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Attaching volume 5601ca28-6c8e-4411-ab14-a96cc9daf3bf to instance 2cc2c98e-adb8-4276-84db-525dafb76698 at mountpoint /dev/vdb on host None. >2018-08-31 04:49:52.304 288514 INFO cinder.volume.manager [req-d094220d-fb23-4dd8-b6c5-3ebd0b1fcfc8 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Attach volume completed successfully. >2018-08-31 04:49:54.087 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-1827a48e-2d87-4e69-b801-7e4e888beef9 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Unable to get remote copy information for volume volume-5601ca28-6c8e-4411-ab14-a96cc9daf3bf >2018-08-31 04:49:55.583 288514 INFO cinder.compute.nova [req-1827a48e-2d87-4e69-b801-7e4e888beef9 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Nova event response: {u'status': u'completed', u'tag': u'5601ca28-6c8e-4411-ab14-a96cc9daf3bf', u'name': u'volume-extended', u'server_uuid': u'2cc2c98e-adb8-4276-84db-525dafb76698', u'code': 200}. >2018-08-31 04:49:55.608 288514 INFO cinder.volume.manager [req-1827a48e-2d87-4e69-b801-7e4e888beef9 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Extend volume completed successfully. >2018-08-31 04:50:23.286 288514 INFO cinder.volume.manager [req-ca4919e3-822d-4723-a7f9-42337a22815c 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Terminate volume connection completed successfully. >2018-08-31 04:50:23.652 288514 INFO cinder.volume.manager [req-9b170d38-2b8b-4002-9de3-390b132cbc42 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Detaching volume 5601ca28-6c8e-4411-ab14-a96cc9daf3bf from instance 2cc2c98e-adb8-4276-84db-525dafb76698. >2018-08-31 04:50:23.733 288514 INFO cinder.volume.manager [req-9b170d38-2b8b-4002-9de3-390b132cbc42 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Detach volume completed successfully. >2018-08-31 04:50:28.711 288514 INFO cinder.volume.manager [req-8b70fef8-f3ef-409d-9cd3-0b91d228e10c 6eced9cc66574ad1816c8ad26318e621 79296b424d114ea294bd5636b7fa06b7 - default default] Deleted volume successfully. >2018-08-31 04:58:52.346 288514 INFO cinder.volume.flows.manager.create_volume [req-3be04f64-c441-4e96-a114-c1894b32bae2 6a4654098c134a0bb016569d40762878 e48bd1a6250a4437b5d7a4c6a1c08c29 - default default] Volume 7e735a65-e427-43d3-aab7-50cfa91cb209: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7e735a65-e427-43d3-aab7-50cfa91cb209'} >2018-08-31 04:58:53.668 288514 INFO cinder.volume.flows.manager.create_volume [req-3be04f64-c441-4e96-a114-c1894b32bae2 6a4654098c134a0bb016569d40762878 e48bd1a6250a4437b5d7a4c6a1c08c29 - default default] Volume volume-7e735a65-e427-43d3-aab7-50cfa91cb209 (7e735a65-e427-43d3-aab7-50cfa91cb209): created successfully >2018-08-31 04:58:53.671 288514 INFO cinder.volume.manager [req-3be04f64-c441-4e96-a114-c1894b32bae2 6a4654098c134a0bb016569d40762878 e48bd1a6250a4437b5d7a4c6a1c08c29 - default default] Created volume successfully. >2018-08-31 04:58:56.393 288514 INFO cinder.volume.manager [req-928f6165-a288-4cc7-8e0d-6f65330aab36 6a4654098c134a0bb016569d40762878 e48bd1a6250a4437b5d7a4c6a1c08c29 - default default] Deleted volume successfully. >2018-08-31 04:59:04.429 288514 INFO cinder.volume.flows.manager.create_volume [req-c887b060-f42b-48f3-930b-d2d28298f437 bfc9d5f02224496ea60126cfda79bc1d eb62bb81ac9e4f9ba8c218f9137c4ef9 - default default] Volume ecd468fa-de1b-44b2-bd4f-0edc938c7825: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-ecd468fa-de1b-44b2-bd4f-0edc938c7825'} >2018-08-31 04:59:06.099 288514 INFO cinder.volume.flows.manager.create_volume [req-c887b060-f42b-48f3-930b-d2d28298f437 bfc9d5f02224496ea60126cfda79bc1d eb62bb81ac9e4f9ba8c218f9137c4ef9 - default default] Volume volume-ecd468fa-de1b-44b2-bd4f-0edc938c7825 (ecd468fa-de1b-44b2-bd4f-0edc938c7825): created successfully >2018-08-31 04:59:06.104 288514 INFO cinder.volume.manager [req-c887b060-f42b-48f3-930b-d2d28298f437 bfc9d5f02224496ea60126cfda79bc1d eb62bb81ac9e4f9ba8c218f9137c4ef9 - default default] Created volume successfully. >2018-08-31 04:59:10.133 288514 WARNING cinder.quota [req-01f9e415-aad0-430a-871d-535822c7d834 bfc9d5f02224496ea60126cfda79bc1d eb62bb81ac9e4f9ba8c218f9137c4ef9 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-189114340 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-189114340, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 04:59:10.134 288514 WARNING cinder.quota [req-01f9e415-aad0-430a-871d-535822c7d834 bfc9d5f02224496ea60126cfda79bc1d eb62bb81ac9e4f9ba8c218f9137c4ef9 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-189114340 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-189114340, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 04:59:10.581 288514 INFO cinder.volume.manager [req-01f9e415-aad0-430a-871d-535822c7d834 bfc9d5f02224496ea60126cfda79bc1d eb62bb81ac9e4f9ba8c218f9137c4ef9 - default default] Deleted volume successfully. >2018-08-31 04:59:58.056 288514 INFO cinder.volume.flows.manager.create_volume [req-156588fa-d05d-46bb-ada7-9d5c0dcaee49 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume a770a654-f044-45cf-9269-6c375f4b7586: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a770a654-f044-45cf-9269-6c375f4b7586'} >2018-08-31 04:59:59.452 288514 INFO cinder.volume.flows.manager.create_volume [req-156588fa-d05d-46bb-ada7-9d5c0dcaee49 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume volume-a770a654-f044-45cf-9269-6c375f4b7586 (a770a654-f044-45cf-9269-6c375f4b7586): created successfully >2018-08-31 04:59:59.455 288514 INFO cinder.volume.manager [req-156588fa-d05d-46bb-ada7-9d5c0dcaee49 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Created volume successfully. >2018-08-31 04:59:59.969 288514 INFO cinder.volume.manager [req-b06db416-8746-4c44-b4a1-3fdfac42cbf7 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Attaching volume a770a654-f044-45cf-9269-6c375f4b7586 to instance b3515905-c432-4067-bbf2-d7c203fa4238 at mountpoint /dev/vdb on host None. >2018-08-31 05:00:00.042 288514 INFO cinder.volume.manager [req-b06db416-8746-4c44-b4a1-3fdfac42cbf7 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Attach volume completed successfully. >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager [req-353b2e45-0e4d-4285-9d07-1c34eb1e4835 c786615be3c14670a190e267e8db6a3b 23d09a5c1e2943e19dba0ce2f8f8652d - default default] Terminate volume connection failed: argument of type 'NoneType' is not iterable: TypeError: argument of type 'NoneType' is not iterable >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager Traceback (most recent call last): >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1768, in terminate_connection >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager force=force) >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_iscsi.py", line 311, in terminate_connection >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager host = connector['host'] if 'host' in connector else "" >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager TypeError: argument of type 'NoneType' is not iterable >2018-08-31 05:00:00.917 288514 ERROR cinder.volume.manager >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server [req-353b2e45-0e4d-4285-9d07-1c34eb1e4835 c786615be3c14670a190e267e8db6a3b 23d09a5c1e2943e19dba0ce2f8f8652d - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server Traceback (most recent call last): >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1773, in terminate_connection >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg) >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: argument of type 'NoneType' is not iterable >2018-08-31 05:00:00.930 288514 ERROR oslo_messaging.rpc.server >2018-08-31 05:00:01.546 288514 INFO cinder.volume.manager [req-9fda7f0c-37d3-46aa-9401-dec07646169a d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Detaching volume a770a654-f044-45cf-9269-6c375f4b7586 from instance b3515905-c432-4067-bbf2-d7c203fa4238. >2018-08-31 05:00:01.632 288514 INFO cinder.volume.manager [req-9fda7f0c-37d3-46aa-9401-dec07646169a d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Detach volume completed successfully. >2018-08-31 05:00:04.041 288753 INFO cinder.volume.flows.manager.create_volume [req-e85192e8-20ec-424a-8cb6-2cd5bcbdb521 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume fc7cea06-7ea7-4f76-8a1e-b46d59e9ebe2: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-fc7cea06-7ea7-4f76-8a1e-b46d59e9ebe2'} >2018-08-31 05:00:05.316 288753 INFO cinder.volume.flows.manager.create_volume [req-e85192e8-20ec-424a-8cb6-2cd5bcbdb521 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume volume-fc7cea06-7ea7-4f76-8a1e-b46d59e9ebe2 (fc7cea06-7ea7-4f76-8a1e-b46d59e9ebe2): created successfully >2018-08-31 05:00:05.320 288753 INFO cinder.volume.manager [req-e85192e8-20ec-424a-8cb6-2cd5bcbdb521 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Created volume successfully. >2018-08-31 05:00:09.056 288753 INFO cinder.volume.manager [req-6e268256-0f5b-42cc-a5a8-e4a920d69bae c786615be3c14670a190e267e8db6a3b 23d09a5c1e2943e19dba0ce2f8f8652d - default default] Deleted volume successfully. >2018-08-31 05:00:09.279 288514 INFO cinder.volume.flows.manager.create_volume [req-89bdb4c1-0cec-49d1-b4b6-c2d7864ba7fd d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume a4dd8f0a-06a0-491d-9bdd-a34d88b62504: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a4dd8f0a-06a0-491d-9bdd-a34d88b62504'} >2018-08-31 05:00:10.952 288514 INFO cinder.volume.flows.manager.create_volume [req-89bdb4c1-0cec-49d1-b4b6-c2d7864ba7fd d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume volume-a4dd8f0a-06a0-491d-9bdd-a34d88b62504 (a4dd8f0a-06a0-491d-9bdd-a34d88b62504): created successfully >2018-08-31 05:00:10.956 288514 INFO cinder.volume.manager [req-89bdb4c1-0cec-49d1-b4b6-c2d7864ba7fd d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Created volume successfully. >2018-08-31 05:00:13.553 288514 INFO cinder.volume.manager [req-059c0d64-d71c-4737-be03-640bb8861aef c786615be3c14670a190e267e8db6a3b 23d09a5c1e2943e19dba0ce2f8f8652d - default default] Deleted volume successfully. >2018-08-31 05:00:14.240 288514 INFO cinder.volume.flows.manager.create_volume [req-13825b65-00ea-41a3-af48-d69ce62451e1 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume 933ef610-5210-4611-a29c-5e4866176e56: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-933ef610-5210-4611-a29c-5e4866176e56'} >2018-08-31 05:00:15.546 288514 INFO cinder.volume.flows.manager.create_volume [req-13825b65-00ea-41a3-af48-d69ce62451e1 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume volume-933ef610-5210-4611-a29c-5e4866176e56 (933ef610-5210-4611-a29c-5e4866176e56): created successfully >2018-08-31 05:00:15.551 288514 INFO cinder.volume.manager [req-13825b65-00ea-41a3-af48-d69ce62451e1 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Created volume successfully. >2018-08-31 05:00:18.857 288514 INFO cinder.volume.manager [req-c3923334-a8c9-44d9-9e75-d93ec70eb536 c786615be3c14670a190e267e8db6a3b 23d09a5c1e2943e19dba0ce2f8f8652d - default default] Deleted volume successfully. >2018-08-31 05:00:19.690 288514 INFO cinder.volume.flows.manager.create_volume [req-bec0300e-e196-472a-af0d-34b39f3d5fc9 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume 9703a82b-5640-4e36-9faf-0a9edc7ad6ef: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9703a82b-5640-4e36-9faf-0a9edc7ad6ef'} >2018-08-31 05:00:20.980 288514 INFO cinder.volume.flows.manager.create_volume [req-bec0300e-e196-472a-af0d-34b39f3d5fc9 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume volume-9703a82b-5640-4e36-9faf-0a9edc7ad6ef (9703a82b-5640-4e36-9faf-0a9edc7ad6ef): created successfully >2018-08-31 05:00:20.984 288514 INFO cinder.volume.manager [req-bec0300e-e196-472a-af0d-34b39f3d5fc9 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Created volume successfully. >2018-08-31 05:00:24.087 288514 INFO cinder.volume.manager [req-519145ad-7a75-40b9-80a5-f0b7e7e4fcf0 c786615be3c14670a190e267e8db6a3b 23d09a5c1e2943e19dba0ce2f8f8652d - default default] Deleted volume successfully. >2018-08-31 05:00:24.721 288514 INFO cinder.volume.flows.manager.create_volume [req-4bbad4ac-1a74-4475-bfea-808e780b5214 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume 1a6f9516-a3b3-43b7-8a03-e113772bc7ca: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1a6f9516-a3b3-43b7-8a03-e113772bc7ca'} >2018-08-31 05:00:25.995 288514 INFO cinder.volume.flows.manager.create_volume [req-4bbad4ac-1a74-4475-bfea-808e780b5214 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Volume volume-1a6f9516-a3b3-43b7-8a03-e113772bc7ca (1a6f9516-a3b3-43b7-8a03-e113772bc7ca): created successfully >2018-08-31 05:00:25.999 288514 INFO cinder.volume.manager [req-4bbad4ac-1a74-4475-bfea-808e780b5214 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Created volume successfully. >2018-08-31 05:00:31.050 288514 INFO cinder.volume.manager [req-d756663b-e6a9-491b-b768-25cc833fbaa7 d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Deleted volume successfully. >2018-08-31 05:00:31.353 288514 INFO cinder.volume.manager [req-5679eeed-6a17-4d2b-b2d8-459206e5894f d43e8e5c57054420960853d22727bbc0 63aa31ad6e66436e8857834994da3267 - default default] Deleted volume successfully. >2018-08-31 05:00:37.095 288514 INFO cinder.volume.flows.manager.create_volume [req-a73eaed2-b64d-4574-b559-f8d262c1f2eb 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Volume f0582f5e-fdaf-4d5c-b762-0b80ef84ecee: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f0582f5e-fdaf-4d5c-b762-0b80ef84ecee'} >2018-08-31 05:00:38.406 288514 INFO cinder.volume.flows.manager.create_volume [req-a73eaed2-b64d-4574-b559-f8d262c1f2eb 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Volume volume-f0582f5e-fdaf-4d5c-b762-0b80ef84ecee (f0582f5e-fdaf-4d5c-b762-0b80ef84ecee): created successfully >2018-08-31 05:00:38.409 288514 INFO cinder.volume.manager [req-a73eaed2-b64d-4574-b559-f8d262c1f2eb 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Created volume successfully. >2018-08-31 05:00:39.340 288514 INFO cinder.volume.flows.manager.create_volume [req-7d32a0ca-23d3-49a1-b3e3-e0446fe67a89 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Volume 0ef46015-07a5-489e-9924-b2624a31c118: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0ef46015-07a5-489e-9924-b2624a31c118'} >2018-08-31 05:00:40.621 288514 INFO cinder.volume.flows.manager.create_volume [req-7d32a0ca-23d3-49a1-b3e3-e0446fe67a89 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Volume volume-0ef46015-07a5-489e-9924-b2624a31c118 (0ef46015-07a5-489e-9924-b2624a31c118): created successfully >2018-08-31 05:00:40.625 288514 INFO cinder.volume.manager [req-7d32a0ca-23d3-49a1-b3e3-e0446fe67a89 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Created volume successfully. >2018-08-31 05:00:42.226 288753 INFO cinder.volume.flows.manager.create_volume [req-47b0c784-3933-4f34-af36-0c2856b5e4a0 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Volume efa5eb16-6296-4788-8862-414763e60444: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-efa5eb16-6296-4788-8862-414763e60444'} >2018-08-31 05:00:43.520 288753 INFO cinder.volume.flows.manager.create_volume [req-47b0c784-3933-4f34-af36-0c2856b5e4a0 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Volume volume-efa5eb16-6296-4788-8862-414763e60444 (efa5eb16-6296-4788-8862-414763e60444): created successfully >2018-08-31 05:00:43.524 288753 INFO cinder.volume.manager [req-47b0c784-3933-4f34-af36-0c2856b5e4a0 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Created volume successfully. >2018-08-31 05:00:45.776 288514 INFO cinder.volume.flows.manager.create_volume [req-1c248cf5-0fa7-4038-825a-8606b15dd4a5 83460242ff23476e856f9f01b5695390 a559188ea7f748fd9953886551110578 - default default] Volume 482c4cb7-d68d-4115-bd6c-c65736195e4d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-482c4cb7-d68d-4115-bd6c-c65736195e4d'} >2018-08-31 05:00:47.067 288514 INFO cinder.volume.flows.manager.create_volume [req-1c248cf5-0fa7-4038-825a-8606b15dd4a5 83460242ff23476e856f9f01b5695390 a559188ea7f748fd9953886551110578 - default default] Volume volume-482c4cb7-d68d-4115-bd6c-c65736195e4d (482c4cb7-d68d-4115-bd6c-c65736195e4d): created successfully >2018-08-31 05:00:47.071 288514 INFO cinder.volume.manager [req-1c248cf5-0fa7-4038-825a-8606b15dd4a5 83460242ff23476e856f9f01b5695390 a559188ea7f748fd9953886551110578 - default default] Created volume successfully. >2018-08-31 05:00:53.901 288514 INFO cinder.volume.manager [req-9949e710-80be-4d17-8964-89c6853f5438 83460242ff23476e856f9f01b5695390 a559188ea7f748fd9953886551110578 - default default] Deleted volume successfully. >2018-08-31 05:00:53.943 288514 INFO cinder.volume.manager [req-ca855633-2a45-4d82-9176-844164c541f2 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Deleted volume successfully. >2018-08-31 05:00:53.988 288514 INFO cinder.volume.manager [req-58a705b8-a5b7-4750-8cd5-f69b48a5fd9c 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Deleted volume successfully. >2018-08-31 05:00:54.222 288753 INFO cinder.volume.manager [req-fa9d2320-c9de-4e3d-af4c-fce71f048dba 4187c1f855d5496fbe03dc9f39948c73 56d76826678c4f55a2bf71692fdc70a1 - default default] Deleted volume successfully. >2018-08-31 05:00:58.199 288514 INFO cinder.volume.flows.manager.create_volume [req-b67bb648-cbeb-495c-9590-a3ae5a82f9bc e5df135380d34f6883ac1775eb073d40 5d99fc6f1be64e178ce42313db1209b4 - default default] Volume 37726558-6a06-42e6-ad99-52b2a5fc12d4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-37726558-6a06-42e6-ad99-52b2a5fc12d4'} >2018-08-31 05:00:59.489 288514 INFO cinder.volume.flows.manager.create_volume [req-b67bb648-cbeb-495c-9590-a3ae5a82f9bc e5df135380d34f6883ac1775eb073d40 5d99fc6f1be64e178ce42313db1209b4 - default default] Volume volume-37726558-6a06-42e6-ad99-52b2a5fc12d4 (37726558-6a06-42e6-ad99-52b2a5fc12d4): created successfully >2018-08-31 05:00:59.492 288514 INFO cinder.volume.manager [req-b67bb648-cbeb-495c-9590-a3ae5a82f9bc e5df135380d34f6883ac1775eb073d40 5d99fc6f1be64e178ce42313db1209b4 - default default] Created volume successfully. >2018-08-31 05:01:06.059 288514 INFO cinder.volume.manager [req-19354972-c322-4d58-9098-0244ca4f1823 e5df135380d34f6883ac1775eb073d40 5d99fc6f1be64e178ce42313db1209b4 - default default] Deleted volume successfully. >2018-08-31 05:01:12.033 288514 INFO cinder.volume.flows.manager.create_volume [req-173538fc-26e1-4de1-ac0d-7bfa76c5a811 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] Volume d69e0c93-b85e-43e3-8dca-90d90f920729: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d69e0c93-b85e-43e3-8dca-90d90f920729'} >2018-08-31 05:01:13.583 288514 INFO cinder.volume.flows.manager.create_volume [req-173538fc-26e1-4de1-ac0d-7bfa76c5a811 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] Volume volume-d69e0c93-b85e-43e3-8dca-90d90f920729 (d69e0c93-b85e-43e3-8dca-90d90f920729): created successfully >2018-08-31 05:01:13.586 288514 INFO cinder.volume.manager [req-173538fc-26e1-4de1-ac0d-7bfa76c5a811 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] Created volume successfully. >2018-08-31 05:01:16.101 288514 INFO cinder.volume.manager [req-c4c44dc3-a86e-42c7-898e-7b2d1891a572 2480dab94305475ebea44c1d906a837e c1317f773ab34445826b0c27e0a8e65d - default default] Transfer volume completed successfully. >2018-08-31 05:01:19.119 288514 INFO cinder.volume.manager [req-9adb93a4-0f8e-44bc-8692-9baf9f116994 5f5c99bc32d9495caf6ac373a41f019b 85caf7b6a43e4a32b1fd027c2c968a00 - default default] Deleted volume successfully. >2018-08-31 05:01:20.375 288514 INFO cinder.volume.flows.manager.create_volume [req-bfc76552-08fc-4a43-abb4-11f835d396fa 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] Volume 8ce773de-3cbd-4aaf-bd25-94719680becc: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-8ce773de-3cbd-4aaf-bd25-94719680becc'} >2018-08-31 05:01:21.873 288514 INFO cinder.volume.flows.manager.create_volume [req-bfc76552-08fc-4a43-abb4-11f835d396fa 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] Volume volume-8ce773de-3cbd-4aaf-bd25-94719680becc (8ce773de-3cbd-4aaf-bd25-94719680becc): created successfully >2018-08-31 05:01:21.877 288514 INFO cinder.volume.manager [req-bfc76552-08fc-4a43-abb4-11f835d396fa 63c2df9d005b4d3b9202a74a445afef6 75927dec5e30477b98444247e667edac - default default] Created volume successfully. >2018-08-31 05:01:25.843 288514 INFO cinder.volume.manager [req-d4839797-76bc-4e9d-8a72-81e6b4bcd859 5f5c99bc32d9495caf6ac373a41f019b 85caf7b6a43e4a32b1fd027c2c968a00 - default default] Deleted volume successfully. >2018-08-31 05:01:30.892 288514 INFO cinder.volume.flows.manager.create_volume [req-28983b70-5f4d-4af9-b26d-17203b3808c8 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Volume d029c6b1-70b9-40e7-ba05-15a1888ebc00: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d029c6b1-70b9-40e7-ba05-15a1888ebc00'} >2018-08-31 05:01:32.183 288514 INFO cinder.volume.flows.manager.create_volume [req-28983b70-5f4d-4af9-b26d-17203b3808c8 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Volume volume-d029c6b1-70b9-40e7-ba05-15a1888ebc00 (d029c6b1-70b9-40e7-ba05-15a1888ebc00): created successfully >2018-08-31 05:01:32.188 288514 INFO cinder.volume.manager [req-28983b70-5f4d-4af9-b26d-17203b3808c8 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Created volume successfully. >2018-08-31 05:01:39.245 288514 INFO cinder.volume.manager [req-53df7c3a-d783-449c-ab50-c0b72948ef6b cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Attaching volume d029c6b1-70b9-40e7-ba05-15a1888ebc00 to instance 5e1e8592-ff90-4588-9122-1ed0cf33789f at mountpoint /dev/vdb on host None. >2018-08-31 05:01:39.319 288514 INFO cinder.volume.manager [req-53df7c3a-d783-449c-ab50-c0b72948ef6b cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Attach volume completed successfully. >2018-08-31 05:01:39.955 288514 INFO cinder.volume.manager [req-40b116b7-ca42-46d7-bdcf-75de4b30434d cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Detaching volume d029c6b1-70b9-40e7-ba05-15a1888ebc00 from instance 5e1e8592-ff90-4588-9122-1ed0cf33789f. >2018-08-31 05:01:40.039 288514 INFO cinder.volume.manager [req-40b116b7-ca42-46d7-bdcf-75de4b30434d cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Detach volume completed successfully. >2018-08-31 05:01:47.608 288514 INFO cinder.volume.manager [req-6259dbda-eecc-4a00-a0e7-ffb1981aed2a cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Attaching volume d029c6b1-70b9-40e7-ba05-15a1888ebc00 to instance daf74105-a37e-4def-95e3-20a69a239402 at mountpoint /dev/vdb on host None. >2018-08-31 05:01:47.679 288514 INFO cinder.volume.manager [req-6259dbda-eecc-4a00-a0e7-ffb1981aed2a cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Attach volume completed successfully. >2018-08-31 05:01:48.418 288514 INFO cinder.volume.manager [req-b73b9baf-5e01-486b-a75b-6c3571b4f327 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Detaching volume d029c6b1-70b9-40e7-ba05-15a1888ebc00 from instance daf74105-a37e-4def-95e3-20a69a239402. >2018-08-31 05:01:48.508 288514 INFO cinder.volume.manager [req-b73b9baf-5e01-486b-a75b-6c3571b4f327 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Detach volume completed successfully. >2018-08-31 05:01:57.624 288514 INFO os_brick.initiator.connectors.iscsi [req-d5d16f85-de32-4a65-ab85-e19f580f8914 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 05:02:23.356 288514 INFO cinder.volume.manager [req-d5d16f85-de32-4a65-ab85-e19f580f8914 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Copy volume to image completed successfully. >2018-08-31 05:02:27.656 288514 INFO cinder.volume.manager [req-2c4df475-e913-4d21-a0a4-9f1fa065e0d2 cdec25ec68974861bd2c5034719003c9 ea14b90b0c8149f1b0dcdd2989abb134 - default default] Deleted volume successfully. >2018-08-31 05:02:33.737 288514 INFO cinder.volume.flows.manager.create_volume [req-cb74ec8d-f612-4c08-95f0-de37b38ee4d0 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Volume 01133abb-4dde-4fe6-9766-5e820fc96854: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-01133abb-4dde-4fe6-9766-5e820fc96854'} >2018-08-31 05:02:35.027 288514 INFO cinder.volume.flows.manager.create_volume [req-cb74ec8d-f612-4c08-95f0-de37b38ee4d0 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Volume volume-01133abb-4dde-4fe6-9766-5e820fc96854 (01133abb-4dde-4fe6-9766-5e820fc96854): created successfully >2018-08-31 05:02:35.031 288514 INFO cinder.volume.manager [req-cb74ec8d-f612-4c08-95f0-de37b38ee4d0 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Created volume successfully. >2018-08-31 05:02:49.364 288514 INFO cinder.volume.manager [req-388463f3-ee1d-4bda-bcf2-8390e589d45b 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Initialize volume connection completed successfully. >2018-08-31 05:02:58.425 288514 INFO cinder.volume.manager [req-172de4ca-2728-4d5e-ad06-eee1dae74433 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Attaching volume 01133abb-4dde-4fe6-9766-5e820fc96854 to instance 4b90e11f-c9ce-4355-80c4-88f70f4ed13e at mountpoint /dev/vdb on host None. >2018-08-31 05:02:58.502 288514 INFO cinder.volume.manager [req-172de4ca-2728-4d5e-ad06-eee1dae74433 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Attach volume completed successfully. >2018-08-31 05:02:59.822 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-57ff0430-448e-46ce-8f1c-4fb7b08f09f5 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Unable to get remote copy information for volume volume-01133abb-4dde-4fe6-9766-5e820fc96854 >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova [req-57ff0430-448e-46ce-8f1c-4fb7b08f09f5 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Failed to notify nova on events: [{'tag': '01133abb-4dde-4fe6-9766-5e820fc96854', 'name': 'volume-extended', 'server_uuid': '4b90e11f-c9ce-4355-80c4-88f70f4ed13e'}].: Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-e997725b-39e0-4ef5-96ac-26d962ae3918) >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova Traceback (most recent call last): >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 182, in _send_events >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova response = nova.server_external_events.create(events) >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/v2/server_external_events.py", line 39, in create >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova return_raw=True) >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova raise exceptions.from_response(resp, body, url, method) >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova Forbidden: Policy doesn't allow os_compute_api:os-server-external-events:create to be performed. (HTTP 403) (Request-ID: req-e997725b-39e0-4ef5-96ac-26d962ae3918) >2018-08-31 05:03:01.319 288514 ERROR cinder.compute.nova >2018-08-31 05:03:01.343 288514 INFO cinder.volume.manager [req-57ff0430-448e-46ce-8f1c-4fb7b08f09f5 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Extend volume completed successfully. >2018-08-31 05:08:13.692 288514 INFO cinder.volume.manager [req-5cefdef9-0f8e-48ff-8462-62a29f9902dc 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Terminate volume connection completed successfully. >2018-08-31 05:08:13.869 288514 INFO cinder.volume.manager [req-bee5751e-1e65-446c-b05b-97372434b15e 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Detaching volume 01133abb-4dde-4fe6-9766-5e820fc96854 from instance 4b90e11f-c9ce-4355-80c4-88f70f4ed13e. >2018-08-31 05:08:13.956 288514 INFO cinder.volume.manager [req-bee5751e-1e65-446c-b05b-97372434b15e 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Detach volume completed successfully. >2018-08-31 05:08:19.657 288514 INFO cinder.volume.manager [req-37f17d19-1beb-417d-994c-5363941e3afd 0b4b336aec7d4227a6f8d57c79c2e607 3e660087da9b4f4da88410c6987d6ca3 - default default] Deleted volume successfully. >2018-08-31 05:08:23.698 288514 INFO cinder.volume.flows.manager.create_volume [req-69264d1f-0d37-404f-aa03-8f5631db012a 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] Volume 2419db78-420a-465a-b960-8cb84ddc0f4f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2419db78-420a-465a-b960-8cb84ddc0f4f'} >2018-08-31 05:08:24.986 288514 INFO cinder.volume.flows.manager.create_volume [req-69264d1f-0d37-404f-aa03-8f5631db012a 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] Volume volume-2419db78-420a-465a-b960-8cb84ddc0f4f (2419db78-420a-465a-b960-8cb84ddc0f4f): created successfully >2018-08-31 05:08:24.991 288514 INFO cinder.volume.manager [req-69264d1f-0d37-404f-aa03-8f5631db012a 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] Created volume successfully. >2018-08-31 05:08:25.860 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-a83a6df0-92e3-40e1-8c69-1480c369bfe3 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] Unable to get remote copy information for volume volume-2419db78-420a-465a-b960-8cb84ddc0f4f >2018-08-31 05:08:26.910 288514 INFO cinder.volume.manager [req-a83a6df0-92e3-40e1-8c69-1480c369bfe3 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] Extend volume completed successfully. >2018-08-31 05:08:30.654 288514 INFO cinder.volume.manager [req-1065e579-ab8f-4fb3-8c81-b4fa82e66b31 8110d82110be4c1e915d96b563792d26 74e8cff3f054424aba89acdcf20f3d54 - default default] Deleted volume successfully. >2018-08-31 05:08:34.328 288514 INFO cinder.volume.flows.manager.create_volume [req-fcbece93-3ee0-4712-998f-07157d9f8d39 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume 0dedfcf2-c91d-426f-9951-738b1a0c8136: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-0dedfcf2-c91d-426f-9951-738b1a0c8136'} >2018-08-31 05:08:35.817 288514 INFO cinder.volume.flows.manager.create_volume [req-fcbece93-3ee0-4712-998f-07157d9f8d39 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume volume-0dedfcf2-c91d-426f-9951-738b1a0c8136 (0dedfcf2-c91d-426f-9951-738b1a0c8136): created successfully >2018-08-31 05:08:35.820 288514 INFO cinder.volume.manager [req-fcbece93-3ee0-4712-998f-07157d9f8d39 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Created volume successfully. >2018-08-31 05:08:38.357 288514 INFO cinder.volume.flows.manager.create_volume [req-81d79489-6d82-4fd1-b337-1506aacff00d 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume d482a8e6-35b7-4f65-8a25-157c547e83e1: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d482a8e6-35b7-4f65-8a25-157c547e83e1'} >2018-08-31 05:08:39.656 288514 INFO cinder.volume.flows.manager.create_volume [req-81d79489-6d82-4fd1-b337-1506aacff00d 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume volume-d482a8e6-35b7-4f65-8a25-157c547e83e1 (d482a8e6-35b7-4f65-8a25-157c547e83e1): created successfully >2018-08-31 05:08:39.660 288514 INFO cinder.volume.manager [req-81d79489-6d82-4fd1-b337-1506aacff00d 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Created volume successfully. >2018-08-31 05:08:42.069 288514 INFO cinder.volume.manager [req-4800f517-99c8-45ed-994b-088bacd666fe 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Deleted volume successfully. >2018-08-31 05:08:44.717 288514 INFO cinder.volume.manager [req-750c3a45-390b-4a66-b528-b17e39204b4d 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Deleted volume successfully. >2018-08-31 05:08:45.508 288514 INFO cinder.volume.flows.manager.create_volume [req-f4e8d615-569a-4c3b-87e4-4e1e6d5df33c 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume a52a742f-e1eb-4f12-a75e-52a3d3d8d48a: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a52a742f-e1eb-4f12-a75e-52a3d3d8d48a'} >2018-08-31 05:08:46.790 288514 INFO cinder.volume.flows.manager.create_volume [req-f4e8d615-569a-4c3b-87e4-4e1e6d5df33c 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume volume-a52a742f-e1eb-4f12-a75e-52a3d3d8d48a (a52a742f-e1eb-4f12-a75e-52a3d3d8d48a): created successfully >2018-08-31 05:08:46.794 288514 INFO cinder.volume.manager [req-f4e8d615-569a-4c3b-87e4-4e1e6d5df33c 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Created volume successfully. >2018-08-31 05:08:48.329 288514 INFO cinder.volume.flows.manager.create_volume [req-2a67379c-4a59-408a-8396-4853795502fe 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume 66ca11fd-6237-44e5-8ead-3a15752c0727: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-66ca11fd-6237-44e5-8ead-3a15752c0727', 'source_volid': 'a52a742f-e1eb-4f12-a75e-52a3d3d8d48a', 'source_volstatus': u'available'} >2018-08-31 05:08:51.092 288514 INFO cinder.volume.flows.manager.create_volume [req-2a67379c-4a59-408a-8396-4853795502fe 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume volume-66ca11fd-6237-44e5-8ead-3a15752c0727 (66ca11fd-6237-44e5-8ead-3a15752c0727): created successfully >2018-08-31 05:08:51.096 288514 INFO cinder.volume.manager [req-2a67379c-4a59-408a-8396-4853795502fe 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Created volume successfully. >2018-08-31 05:08:52.503 288514 INFO cinder.volume.flows.manager.create_volume [req-825a6cdd-7473-4edf-a782-58361061caea 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume 7e645473-8806-4add-a3d7-304aea9aeb1d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7e645473-8806-4add-a3d7-304aea9aeb1d'} >2018-08-31 05:08:54.251 288514 INFO cinder.volume.flows.manager.create_volume [req-825a6cdd-7473-4edf-a782-58361061caea 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume volume-7e645473-8806-4add-a3d7-304aea9aeb1d (7e645473-8806-4add-a3d7-304aea9aeb1d): created successfully >2018-08-31 05:08:54.256 288514 INFO cinder.volume.manager [req-825a6cdd-7473-4edf-a782-58361061caea 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Created volume successfully. >2018-08-31 05:08:56.990 288514 INFO cinder.volume.manager [req-b4eceb0b-b630-4967-af54-08a63f162a89 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Deleted volume successfully. >2018-08-31 05:09:02.144 288514 INFO cinder.volume.manager [req-cdefea35-daac-4ea8-8157-70912f6cad36 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Deleted volume successfully. >2018-08-31 05:09:04.230 288514 INFO cinder.volume.flows.manager.create_volume [req-ed1e4c97-954a-419d-be52-dbc0bf981f81 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume 0e4fe73b-e93a-4abf-b644-68cb32816360: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-0e4fe73b-e93a-4abf-b644-68cb32816360', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f636d1a5dd0>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 05:09:05.572 288514 INFO cinder.image.image_utils [req-ed1e4c97-954a-419d-be52-dbc0bf981f81 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Image download 12.00 MB at 8.96 MB/s >2018-08-31 05:09:13.657 288514 INFO os_brick.initiator.connectors.iscsi [req-ed1e4c97-954a-419d-be52-dbc0bf981f81 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Trying to connect to iSCSI portal 9.11.54.170:3260 >2018-08-31 05:09:21.312 288514 INFO cinder.image.image_utils [req-ed1e4c97-954a-419d-be52-dbc0bf981f81 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Converted 39.00 MB image at 39.00 MB/s >2018-08-31 05:09:25.222 288514 INFO cinder.volume.flows.manager.create_volume [req-ed1e4c97-954a-419d-be52-dbc0bf981f81 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume volume-0e4fe73b-e93a-4abf-b644-68cb32816360 (0e4fe73b-e93a-4abf-b644-68cb32816360): created successfully >2018-08-31 05:09:25.226 288514 INFO cinder.volume.manager [req-ed1e4c97-954a-419d-be52-dbc0bf981f81 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Created volume successfully. >2018-08-31 05:09:26.330 288514 INFO cinder.volume.flows.manager.create_volume [req-cd2d6021-28d1-437d-80d7-7a6bc61af7e2 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume 3ca4baf1-7af3-4138-944a-64a5ffffbe51: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-3ca4baf1-7af3-4138-944a-64a5ffffbe51'} >2018-08-31 05:09:27.694 288514 INFO cinder.volume.flows.manager.create_volume [req-cd2d6021-28d1-437d-80d7-7a6bc61af7e2 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Volume volume-3ca4baf1-7af3-4138-944a-64a5ffffbe51 (3ca4baf1-7af3-4138-944a-64a5ffffbe51): created successfully >2018-08-31 05:09:27.698 288514 INFO cinder.volume.manager [req-cd2d6021-28d1-437d-80d7-7a6bc61af7e2 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Created volume successfully. >2018-08-31 05:09:30.785 288514 INFO cinder.volume.manager [req-3d426726-f86f-4bb1-bb5a-06a837004cf1 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Deleted volume successfully. >2018-08-31 05:09:32.875 288514 INFO cinder.volume.manager [req-262e64b6-99c5-44ba-b59e-59429912db70 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Deleted volume successfully. >2018-08-31 05:09:35.294 288514 INFO cinder.volume.manager [req-a9e339ea-74f8-4601-bca9-19bcf7ede2fb 74cdd900e096483d9f99f92259a59341 c124f2dcbf9e476587522966892bd2b3 - default default] Deleted volume successfully. >2018-08-31 05:09:42.101 288514 INFO cinder.volume.flows.manager.create_volume [req-c9f35eeb-37dc-45b9-a19a-7b3c41dd4af4 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Volume d93d4df2-2785-4018-aebe-d86497f48fb4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-d93d4df2-2785-4018-aebe-d86497f48fb4'} >2018-08-31 05:09:43.374 288514 INFO cinder.volume.flows.manager.create_volume [req-c9f35eeb-37dc-45b9-a19a-7b3c41dd4af4 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Volume volume-d93d4df2-2785-4018-aebe-d86497f48fb4 (d93d4df2-2785-4018-aebe-d86497f48fb4): created successfully >2018-08-31 05:09:43.379 288514 INFO cinder.volume.manager [req-c9f35eeb-37dc-45b9-a19a-7b3c41dd4af4 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Created volume successfully. >2018-08-31 05:09:44.195 288753 INFO cinder.volume.flows.manager.create_volume [req-bce754be-0e04-4938-b233-9c741ad052ff b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Volume a2be7e48-02ef-4e08-94a5-ce1c962c2f82: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a2be7e48-02ef-4e08-94a5-ce1c962c2f82'} >2018-08-31 05:09:45.585 288753 INFO cinder.volume.flows.manager.create_volume [req-bce754be-0e04-4938-b233-9c741ad052ff b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Volume volume-a2be7e48-02ef-4e08-94a5-ce1c962c2f82 (a2be7e48-02ef-4e08-94a5-ce1c962c2f82): created successfully >2018-08-31 05:09:45.590 288753 INFO cinder.volume.manager [req-bce754be-0e04-4938-b233-9c741ad052ff b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Created volume successfully. >2018-08-31 05:09:47.335 288514 INFO cinder.volume.flows.manager.create_volume [req-7c47c4c0-e65a-4d10-a3e5-4575c84c53c2 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Volume 835f0b00-2145-4eeb-ae1a-ad96ef4d0b3d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-835f0b00-2145-4eeb-ae1a-ad96ef4d0b3d'} >2018-08-31 05:09:48.842 288514 INFO cinder.volume.flows.manager.create_volume [req-7c47c4c0-e65a-4d10-a3e5-4575c84c53c2 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Volume volume-835f0b00-2145-4eeb-ae1a-ad96ef4d0b3d (835f0b00-2145-4eeb-ae1a-ad96ef4d0b3d): created successfully >2018-08-31 05:09:48.846 288514 INFO cinder.volume.manager [req-7c47c4c0-e65a-4d10-a3e5-4575c84c53c2 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Created volume successfully. >2018-08-31 05:09:57.419 288514 INFO cinder.volume.manager [req-505fcf7f-8013-48d5-bc60-cbcd071e321d b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Deleted volume successfully. >2018-08-31 05:09:57.457 288753 INFO cinder.volume.manager [req-342f21ca-4345-4ec2-8d89-1b7d57d21894 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Deleted volume successfully. >2018-08-31 05:09:57.951 288514 INFO cinder.volume.manager [req-012a04b1-517a-4333-b021-f7de5e1959c9 b0b91201a103469cbbad63130a61d2df dc6b01402a0b4bc991cc66132a4cc5a7 - default default] Deleted volume successfully. >2018-08-31 05:10:00.772 288514 INFO cinder.volume.flows.manager.create_volume [req-77e11511-dcf9-4424-9580-27ae9aca21c0 38481db203b8489eb52f86d9d97ac026 fe034cb24dce43d6bdd745b5b1beee0e - default default] Volume 41710576-ff89-4bc7-a47b-e9f84cb92dd9: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-41710576-ff89-4bc7-a47b-e9f84cb92dd9'} >2018-08-31 05:10:02.108 288514 INFO cinder.volume.flows.manager.create_volume [req-77e11511-dcf9-4424-9580-27ae9aca21c0 38481db203b8489eb52f86d9d97ac026 fe034cb24dce43d6bdd745b5b1beee0e - default default] Volume volume-41710576-ff89-4bc7-a47b-e9f84cb92dd9 (41710576-ff89-4bc7-a47b-e9f84cb92dd9): created successfully >2018-08-31 05:10:02.111 288514 INFO cinder.volume.manager [req-77e11511-dcf9-4424-9580-27ae9aca21c0 38481db203b8489eb52f86d9d97ac026 fe034cb24dce43d6bdd745b5b1beee0e - default default] Created volume successfully. >2018-08-31 05:10:22.379 288514 INFO cinder.volume.manager [req-d7973ea7-031f-49c3-a68e-1006bc8ebd34 38481db203b8489eb52f86d9d97ac026 fe034cb24dce43d6bdd745b5b1beee0e - default default] Deleted volume successfully. >2018-08-31 05:10:26.352 288514 INFO cinder.volume.flows.manager.create_volume [req-86d40059-c779-4376-b09d-1a9999ff9f95 cb2812b7f69f44a1864638fff4d21819 3a945c1a31a14454ade1843c14489e7a - default default] Volume 5349d2d7-7276-4a22-8db6-c2030e08ad8d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-5349d2d7-7276-4a22-8db6-c2030e08ad8d'} >2018-08-31 05:10:27.686 288514 INFO cinder.volume.flows.manager.create_volume [req-86d40059-c779-4376-b09d-1a9999ff9f95 cb2812b7f69f44a1864638fff4d21819 3a945c1a31a14454ade1843c14489e7a - default default] Volume volume-5349d2d7-7276-4a22-8db6-c2030e08ad8d (5349d2d7-7276-4a22-8db6-c2030e08ad8d): created successfully >2018-08-31 05:10:27.690 288514 INFO cinder.volume.manager [req-86d40059-c779-4376-b09d-1a9999ff9f95 cb2812b7f69f44a1864638fff4d21819 3a945c1a31a14454ade1843c14489e7a - default default] Created volume successfully. >2018-08-31 05:10:30.247 288514 INFO cinder.volume.manager [req-9c00ea71-185c-4742-bf38-1c35c01c0498 cb2812b7f69f44a1864638fff4d21819 3a945c1a31a14454ade1843c14489e7a - default default] Deleted volume successfully. >2018-08-31 05:11:18.820 288514 INFO cinder.volume.flows.manager.create_volume [req-d62c5e2e-f935-422e-aeed-6aff47d02d13 e1d61e97880045e7b29dd9c420651261 0d7dbe5b39af4e99b2b8780951c9a094 - default default] Volume c72be9b1-79d9-4fdd-a0b2-8747577aa465: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-c72be9b1-79d9-4fdd-a0b2-8747577aa465'} >2018-08-31 05:11:20.526 288514 INFO cinder.volume.flows.manager.create_volume [req-d62c5e2e-f935-422e-aeed-6aff47d02d13 e1d61e97880045e7b29dd9c420651261 0d7dbe5b39af4e99b2b8780951c9a094 - default default] Volume volume-c72be9b1-79d9-4fdd-a0b2-8747577aa465 (c72be9b1-79d9-4fdd-a0b2-8747577aa465): created successfully >2018-08-31 05:11:20.530 288514 INFO cinder.volume.manager [req-d62c5e2e-f935-422e-aeed-6aff47d02d13 e1d61e97880045e7b29dd9c420651261 0d7dbe5b39af4e99b2b8780951c9a094 - default default] Created volume successfully. >2018-08-31 05:11:24.497 288514 INFO cinder.volume.manager [req-72de3b7d-8963-4b82-b81d-cab087b74da5 e1d61e97880045e7b29dd9c420651261 0d7dbe5b39af4e99b2b8780951c9a094 - default default] Deleted volume successfully. >2018-08-31 05:11:29.790 288514 INFO cinder.volume.flows.manager.create_volume [req-103f93b2-784e-4073-a47f-7333db2e0a5e 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Volume 95371205-f4fc-4300-9008-bd190c827d78: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-95371205-f4fc-4300-9008-bd190c827d78', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f636cd00810>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 05:11:31.316 288514 INFO cinder.image.image_utils [req-103f93b2-784e-4073-a47f-7333db2e0a5e 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Image download 12.00 MB at 7.87 MB/s >2018-08-31 05:11:38.368 288514 INFO os_brick.initiator.connectors.iscsi [req-103f93b2-784e-4073-a47f-7333db2e0a5e 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 05:11:46.066 288514 INFO cinder.image.image_utils [req-103f93b2-784e-4073-a47f-7333db2e0a5e 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Converted 39.00 MB image at 37.87 MB/s >2018-08-31 05:11:49.856 288514 INFO cinder.volume.flows.manager.create_volume [req-103f93b2-784e-4073-a47f-7333db2e0a5e 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Volume volume-95371205-f4fc-4300-9008-bd190c827d78 (95371205-f4fc-4300-9008-bd190c827d78): created successfully >2018-08-31 05:11:49.860 288514 INFO cinder.volume.manager [req-103f93b2-784e-4073-a47f-7333db2e0a5e 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Created volume successfully. >2018-08-31 05:11:51.338 288514 INFO cinder.volume.flows.manager.create_volume [req-bc979446-080d-4014-8a91-5e64ba111405 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Volume 12bc0267-878a-40d0-ac95-5e8a0a98adf0: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-12bc0267-878a-40d0-ac95-5e8a0a98adf0', 'source_volid': '95371205-f4fc-4300-9008-bd190c827d78', 'source_volstatus': u'available'} >2018-08-31 05:11:53.766 288514 INFO cinder.volume.flows.manager.create_volume [req-bc979446-080d-4014-8a91-5e64ba111405 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Volume volume-12bc0267-878a-40d0-ac95-5e8a0a98adf0 (12bc0267-878a-40d0-ac95-5e8a0a98adf0): created successfully >2018-08-31 05:11:53.771 288514 INFO cinder.volume.manager [req-bc979446-080d-4014-8a91-5e64ba111405 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Created volume successfully. >2018-08-31 05:11:59.419 288514 INFO cinder.volume.manager [req-6db27d51-e0d7-4bf3-9fcd-aaf3ff8ca068 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Deleted volume successfully. >2018-08-31 05:12:00.996 288514 INFO cinder.volume.manager [req-2595f430-10fc-4063-8335-00771c6afdb2 3ce84733ad184962b84472580f460e3b d12305a6dd7844b39e037f785fae7bf2 - default default] Deleted volume successfully. >2018-08-31 05:12:04.265 288514 INFO cinder.volume.flows.manager.create_volume [req-e323541d-e822-4691-add1-c6fcf8e4a66a bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Volume 59a7b073-c7ff-4fb2-aa16-d76ecfd01333: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-59a7b073-c7ff-4fb2-aa16-d76ecfd01333'} >2018-08-31 05:12:05.670 288514 INFO cinder.volume.flows.manager.create_volume [req-e323541d-e822-4691-add1-c6fcf8e4a66a bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Volume volume-59a7b073-c7ff-4fb2-aa16-d76ecfd01333 (59a7b073-c7ff-4fb2-aa16-d76ecfd01333): created successfully >2018-08-31 05:12:05.675 288514 INFO cinder.volume.manager [req-e323541d-e822-4691-add1-c6fcf8e4a66a bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Created volume successfully. >2018-08-31 05:12:07.533 288753 INFO cinder.volume.flows.manager.create_volume [req-32b11580-b24b-4138-9e3e-46dab2d1c83e bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Volume bf437ab8-8867-4020-a016-b78cb87d6161: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-bf437ab8-8867-4020-a016-b78cb87d6161'} >2018-08-31 05:12:08.881 288753 INFO cinder.volume.flows.manager.create_volume [req-32b11580-b24b-4138-9e3e-46dab2d1c83e bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Volume volume-bf437ab8-8867-4020-a016-b78cb87d6161 (bf437ab8-8867-4020-a016-b78cb87d6161): created successfully >2018-08-31 05:12:08.885 288753 INFO cinder.volume.manager [req-32b11580-b24b-4138-9e3e-46dab2d1c83e bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Created volume successfully. >2018-08-31 05:12:10.670 288514 INFO cinder.volume.flows.manager.create_volume [req-d7122d07-6edc-49fb-b32e-c6f9c55b25f4 bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Volume 31c7438a-4082-4eaa-a5ec-6b787458edf0: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-31c7438a-4082-4eaa-a5ec-6b787458edf0'} >2018-08-31 05:12:11.962 288514 INFO cinder.volume.flows.manager.create_volume [req-d7122d07-6edc-49fb-b32e-c6f9c55b25f4 bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Volume volume-31c7438a-4082-4eaa-a5ec-6b787458edf0 (31c7438a-4082-4eaa-a5ec-6b787458edf0): created successfully >2018-08-31 05:12:11.967 288514 INFO cinder.volume.manager [req-d7122d07-6edc-49fb-b32e-c6f9c55b25f4 bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Created volume successfully. >2018-08-31 05:12:18.394 288514 INFO cinder.volume.manager [req-9cb3de85-9c04-4622-9fca-2024e1f01602 bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Deleted volume successfully. >2018-08-31 05:12:18.412 288753 INFO cinder.volume.manager [req-f8111cba-d1ea-4b10-bfda-e15827092e2c bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Deleted volume successfully. >2018-08-31 05:12:18.656 288514 INFO cinder.volume.manager [req-7c55e32b-004e-4b55-ab70-88a20ef43fbb bd6b390319404389847291c67aa0371a befc94fe3e0f44ceb8731fc3cdb23c38 - default default] Deleted volume successfully. >2018-08-31 05:12:39.327 288514 INFO cinder.volume.flows.manager.create_volume [req-d6cd8853-e901-4f68-bf34-e0e5197fef9e 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Volume 2e7b4da2-c54b-4289-8891-c90a20e78ae3: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3'} >2018-08-31 05:12:40.730 288514 INFO cinder.volume.flows.manager.create_volume [req-d6cd8853-e901-4f68-bf34-e0e5197fef9e 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Volume volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 (2e7b4da2-c54b-4289-8891-c90a20e78ae3): created successfully >2018-08-31 05:12:40.734 288514 INFO cinder.volume.manager [req-d6cd8853-e901-4f68-bf34-e0e5197fef9e 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Created volume successfully. >2018-08-31 05:12:42.062 288753 INFO cinder.volume.flows.manager.create_volume [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Volume a8621daa-62c9-4829-b128-3954fa3f99e4: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-a8621daa-62c9-4829-b128-3954fa3f99e4'} >2018-08-31 05:12:43.397 288753 INFO cinder.volume.flows.manager.create_volume [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Volume volume-a8621daa-62c9-4829-b128-3954fa3f99e4 (a8621daa-62c9-4829-b128-3954fa3f99e4): created successfully >2018-08-31 05:12:43.401 288753 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Created volume successfully. >2018-08-31 05:12:51.395 288753 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Initialize volume connection completed successfully. >2018-08-31 05:12:51.401 288514 INFO os_brick.initiator.connectors.iscsi [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Trying to connect to iSCSI portal 9.11.54.170:3260 >2018-08-31 05:12:58.322 288514 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Initialize volume connection completed successfully. >2018-08-31 05:12:58.323 288514 INFO os_brick.initiator.connectors.iscsi [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 05:13:23.852 288514 INFO cinder.volume.utils [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Volume copy 1024.00 MB at 51.46 MB/s >2018-08-31 05:13:27.370 288753 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Terminate volume connection completed successfully. >2018-08-31 05:13:27.415 288753 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Remove volume export completed successfully. >2018-08-31 05:13:27.599 361223 WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-3600507680c8f806cf0000000000000d1 >2018-08-31 05:13:29.273 288514 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Terminate volume connection completed successfully. >2018-08-31 05:13:29.309 288514 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Remove volume export completed successfully. >2018-08-31 05:13:29.594 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Error has occurred: Unexpected error while running command. >Command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n': ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 05:13:30.066 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Error running SSH command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 05:13:30.067 288753 WARNING cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Not able to use storwize_san_secondary_ip since it is not configured.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 05:13:30.068 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Error running SSH command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4": ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 05:13:30.069 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3', '"volume-a8621daa-62c9-4829-b128-3954fa3f99e4"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >.: ProcessExecutionError: Unexpected error while running command. >Command: svctask chvdisk -name volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3 "volume-a8621daa-62c9-4829-b128-3954fa3f99e4" >Exit code: 1 >Stdout: u'' >Stderr: u'CMMVC6035E The action failed as the object already exists.\n' >2018-08-31 05:13:30.070 288753 ERROR cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Unable to rename the logical volume for volume: 2e7b4da2-c54b-4289-8891-c90a20e78ae3: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output: > command: ['svctask', 'chvdisk', '-name', 'volume-2e7b4da2-c54b-4289-8891-c90a20e78ae3', '"volume-a8621daa-62c9-4829-b128-3954fa3f99e4"'] > stdout: > stderr: CMMVC6035E The action failed as the object already exists. >. >2018-08-31 05:13:30.206 288514 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Complete-Migrate volume completed successfully. >2018-08-31 05:13:30.206 288514 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Migrate volume completed successfully. >2018-08-31 05:13:30.745 288514 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Retype volume completed successfully. >2018-08-31 05:13:32.093 288514 INFO cinder.volume.manager [req-90ba1835-5e44-46fe-8f52-c2f56f795712 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Deleted volume successfully. >2018-08-31 05:13:34.411 288753 WARNING cinder.quota [req-19768a12-1832-49d1-bf49-4c45c8b4b1b3 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Deprecated: Default quota for resource: gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1288938053 is set by the default quota flag: quota_gigabytes_tempest-VolumeRetypeWithMigrationTest-volume-type-1288938053, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 05:13:34.412 288753 WARNING cinder.quota [req-19768a12-1832-49d1-bf49-4c45c8b4b1b3 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Deprecated: Default quota for resource: volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1288938053 is set by the default quota flag: quota_volumes_tempest-VolumeRetypeWithMigrationTest-volume-type-1288938053, it is now deprecated. Please use the default quota class for default quota. >2018-08-31 05:13:35.013 288753 INFO cinder.volume.manager [req-19768a12-1832-49d1-bf49-4c45c8b4b1b3 3e7cff315c334b8d9e2977fb3b8801ff 87b9a95f36ab48b39ac32d57b8a42f09 - default default] Deleted volume successfully. >2018-08-31 05:13:55.743 288753 INFO cinder.volume.flows.manager.create_volume [req-cf76272d-bb33-49f5-a65d-8b46a73ab86b b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Volume 62c113c0-99d3-4f4c-8620-c71752715dfc: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-62c113c0-99d3-4f4c-8620-c71752715dfc'} >2018-08-31 05:13:57.425 288753 INFO cinder.volume.flows.manager.create_volume [req-cf76272d-bb33-49f5-a65d-8b46a73ab86b b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Volume volume-62c113c0-99d3-4f4c-8620-c71752715dfc (62c113c0-99d3-4f4c-8620-c71752715dfc): created successfully >2018-08-31 05:13:57.429 288753 INFO cinder.volume.manager [req-cf76272d-bb33-49f5-a65d-8b46a73ab86b b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Created volume successfully. >2018-08-31 05:14:01.373 288753 INFO cinder.volume.manager [req-1d0d46d0-96ae-4863-9e0d-c8b183856a32 b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Create snapshot completed successfully >2018-08-31 05:14:07.205 288753 INFO cinder.volume.manager [req-0215d570-ab2e-49b2-a796-ff128eaa0401 b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Create snapshot completed successfully >2018-08-31 05:14:11.734 288753 INFO cinder.volume.manager [req-945bc5b6-308a-4500-a605-6df8afaf1e65 a035f7f8510a439ab2bb0fd3ccff0dce d8e066c3a2684f87a617bb22786dfb5e - default default] Delete snapshot completed successfully. >2018-08-31 05:14:15.464 288753 INFO cinder.volume.manager [req-61cce1e1-7056-47bf-84d8-8b9b476b349a b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Create snapshot completed successfully >2018-08-31 05:14:19.161 288753 INFO cinder.volume.manager [req-f1a3ff69-4cdd-472f-89cc-5c8628bea2ac a035f7f8510a439ab2bb0fd3ccff0dce d8e066c3a2684f87a617bb22786dfb5e - default default] Delete snapshot completed successfully. >2018-08-31 05:14:22.301 288753 INFO cinder.volume.manager [req-ec6910cd-d22a-4ac8-93bd-36f5bd61eb45 b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Create snapshot completed successfully >2018-08-31 05:14:26.215 288753 INFO cinder.volume.manager [req-6d363b4c-0652-4824-a028-389f97e77ab8 a035f7f8510a439ab2bb0fd3ccff0dce d8e066c3a2684f87a617bb22786dfb5e - default default] Delete snapshot completed successfully. >2018-08-31 05:14:29.661 288753 INFO cinder.volume.manager [req-a961904f-19ec-403a-a085-1396f76d1052 b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Create snapshot completed successfully >2018-08-31 05:14:33.465 288753 INFO cinder.volume.manager [req-ba43aed1-f458-4bfb-aecd-845040c0e120 a035f7f8510a439ab2bb0fd3ccff0dce d8e066c3a2684f87a617bb22786dfb5e - default default] Delete snapshot completed successfully. >2018-08-31 05:14:37.526 288753 INFO cinder.volume.manager [req-fb3236f9-7431-4fd9-a7d1-83ef2ca33660 b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Delete snapshot completed successfully. >2018-08-31 05:14:40.442 288753 INFO cinder.volume.manager [req-bfa0a2e3-7117-43cf-869a-e28642e36d06 b98dd581ff784943a7d72dd25c3f6fa3 5ac48a7170c3442589ac91162d5364e0 - default default] Deleted volume successfully. >2018-08-31 05:14:44.379 288753 INFO cinder.volume.flows.manager.create_volume [req-92b05395-2480-4b10-8567-b7f482fd9805 f0194726fecd48f184bacc1fa47a8952 dda0cee6b6c6489ea22132e8e490e335 - default default] Volume 71056ef0-d138-47ab-a056-5eec3d8c4838: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-71056ef0-d138-47ab-a056-5eec3d8c4838'} >2018-08-31 05:14:45.721 288753 INFO cinder.volume.flows.manager.create_volume [req-92b05395-2480-4b10-8567-b7f482fd9805 f0194726fecd48f184bacc1fa47a8952 dda0cee6b6c6489ea22132e8e490e335 - default default] Volume volume-71056ef0-d138-47ab-a056-5eec3d8c4838 (71056ef0-d138-47ab-a056-5eec3d8c4838): created successfully >2018-08-31 05:14:45.725 288753 INFO cinder.volume.manager [req-92b05395-2480-4b10-8567-b7f482fd9805 f0194726fecd48f184bacc1fa47a8952 dda0cee6b6c6489ea22132e8e490e335 - default default] Created volume successfully. >2018-08-31 05:14:48.906 288753 INFO cinder.volume.manager [req-0fb700ef-5c69-4bde-9c18-792145a382f0 f0194726fecd48f184bacc1fa47a8952 dda0cee6b6c6489ea22132e8e490e335 - default default] Create snapshot completed successfully >2018-08-31 05:14:54.968 288753 INFO cinder.volume.manager [req-389dfa47-07a2-4fa6-9e6b-7788a5b7c27a f0194726fecd48f184bacc1fa47a8952 dda0cee6b6c6489ea22132e8e490e335 - default default] Delete snapshot completed successfully. >2018-08-31 05:14:57.841 288753 INFO cinder.volume.manager [req-7e8219b3-6fa6-4208-8615-c4f68f91ffa0 f0194726fecd48f184bacc1fa47a8952 dda0cee6b6c6489ea22132e8e490e335 - default default] Deleted volume successfully. >2018-08-31 05:15:01.048 288753 INFO cinder.volume.flows.manager.create_volume [req-0d87a5da-e00c-44ae-88ad-844708c13c3c f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Volume 20073d3e-113d-43a2-97bc-803fb528dfcf: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-20073d3e-113d-43a2-97bc-803fb528dfcf'} >2018-08-31 05:15:02.659 288753 INFO cinder.volume.flows.manager.create_volume [req-0d87a5da-e00c-44ae-88ad-844708c13c3c f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Volume volume-20073d3e-113d-43a2-97bc-803fb528dfcf (20073d3e-113d-43a2-97bc-803fb528dfcf): created successfully >2018-08-31 05:15:02.663 288753 INFO cinder.volume.manager [req-0d87a5da-e00c-44ae-88ad-844708c13c3c f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Created volume successfully. >2018-08-31 05:15:07.015 288753 INFO cinder.volume.manager [req-f86b9ab2-20e0-454a-8d30-ed2dd821cedf f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:15:15.827 288753 INFO cinder.volume.manager [req-739c8983-bc23-46aa-a638-cd5d4847624d f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Initialize volume connection completed successfully. >2018-08-31 05:15:49.288 288753 INFO cinder.volume.manager [req-739c8983-bc23-46aa-a638-cd5d4847624d f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Terminate volume connection completed successfully. >2018-08-31 05:15:49.335 288753 INFO cinder.volume.manager [req-739c8983-bc23-46aa-a638-cd5d4847624d f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Remove volume export completed successfully. >2018-08-31 05:15:54.324 288753 INFO cinder.volume.manager [req-739c8983-bc23-46aa-a638-cd5d4847624d f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Deleted volume successfully. >2018-08-31 05:16:08.360 288753 INFO cinder.volume.manager [req-c0b79f50-8b55-43c2-aeab-aa1683abf096 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Initialize volume connection completed successfully. >2018-08-31 05:16:17.164 288753 INFO cinder.volume.manager [req-19fb8dce-0a10-475f-a24b-91fb0dee2f31 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Attaching volume 20073d3e-113d-43a2-97bc-803fb528dfcf to instance dab42520-fd14-431c-957e-83a482b1081d at mountpoint /dev/vdb on host None. >2018-08-31 05:16:17.357 288753 INFO cinder.volume.manager [req-19fb8dce-0a10-475f-a24b-91fb0dee2f31 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Attach volume completed successfully. >2018-08-31 05:16:20.330 288753 INFO cinder.volume.manager [req-e8714cc5-d899-4182-9bae-5f251474b419 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:16:24.181 288753 INFO cinder.volume.manager [req-6dd04117-5422-4a8f-9c49-3c3d1123a094 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:16:27.430 288753 INFO cinder.volume.manager [req-96770ace-ec78-4e7a-8851-b52e8d53008e f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:16:31.216 288753 INFO cinder.volume.manager [req-9ed32b26-b3d2-410c-97b0-68653d851fe4 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:16:34.818 288753 INFO cinder.volume.manager [req-7bab76a0-0680-4527-9996-80d376d8e0a3 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:16:38.169 288753 INFO cinder.volume.manager [req-e0652f41-3985-4c02-9908-235ce37133cf f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:16:49.060 288753 INFO cinder.volume.manager [req-ab180e44-c899-47ca-a1ad-fdc1983f621d f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Terminate volume connection completed successfully. >2018-08-31 05:16:49.215 288753 INFO cinder.volume.manager [req-8691283a-75aa-4c6e-8a73-42efc6425417 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Detaching volume 20073d3e-113d-43a2-97bc-803fb528dfcf from instance dab42520-fd14-431c-957e-83a482b1081d. >2018-08-31 05:16:49.301 288753 INFO cinder.volume.manager [req-8691283a-75aa-4c6e-8a73-42efc6425417 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Detach volume completed successfully. >2018-08-31 05:16:53.891 288753 INFO cinder.volume.manager [req-a3d001e0-fb3a-4191-9fc0-42a46787c490 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:16:58.317 288753 INFO cinder.volume.manager [req-05324a14-73dc-4126-88dd-e286bd3aaa8a f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:17:03.131 288753 INFO cinder.volume.manager [req-10444c03-9f94-4036-a7a5-3f76bca78792 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:17:16.502 288753 INFO cinder.volume.manager [req-304a7cdf-63f1-4aa2-8a86-cd5d2ce29014 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Initialize volume connection completed successfully. >2018-08-31 05:17:25.153 288753 INFO cinder.volume.manager [req-6ff523a6-9310-412f-992f-ced85866325a f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Attaching volume 20073d3e-113d-43a2-97bc-803fb528dfcf to instance 406e953f-d345-485a-ac46-030c40abb19b at mountpoint /dev/vdb on host None. >2018-08-31 05:17:25.228 288753 INFO cinder.volume.manager [req-6ff523a6-9310-412f-992f-ced85866325a f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Attach volume completed successfully. >2018-08-31 05:17:28.576 288753 INFO cinder.volume.manager [req-31bff222-4aa7-441e-a110-8581e386cb2b f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:17:31.802 288753 INFO cinder.volume.manager [req-3d009e96-602c-4f91-83e7-e129ac7c0e64 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:17:35.195 288753 INFO cinder.volume.manager [req-7130175c-0354-4bca-beb3-8ddbc132c24b f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:17:38.604 288753 INFO cinder.volume.manager [req-9fb85a10-4a63-4d13-a397-983bdcfad2a6 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:17:42.230 288753 INFO cinder.volume.manager [req-908d3b2a-7a68-4e2e-bb2d-dca2aaa90bd9 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:17:52.354 288753 INFO cinder.volume.manager [req-9a68363c-d198-4203-9898-fbaadfbfb2a9 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Terminate volume connection completed successfully. >2018-08-31 05:17:52.498 288753 INFO cinder.volume.manager [req-ae7b0639-0362-4c62-98f0-ad226bd683ac f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Detaching volume 20073d3e-113d-43a2-97bc-803fb528dfcf from instance 406e953f-d345-485a-ac46-030c40abb19b. >2018-08-31 05:17:52.582 288753 INFO cinder.volume.manager [req-ae7b0639-0362-4c62-98f0-ad226bd683ac f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Detach volume completed successfully. >2018-08-31 05:17:54.795 288514 INFO cinder.volume.flows.manager.create_volume [req-f8828022-4d84-4f39-826f-d519b3eb1345 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Volume 04ef286e-3020-4895-91ef-583b185597ad: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-04ef286e-3020-4895-91ef-583b185597ad'} >2018-08-31 05:17:56.115 288514 INFO cinder.volume.flows.manager.create_volume [req-f8828022-4d84-4f39-826f-d519b3eb1345 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Volume volume-04ef286e-3020-4895-91ef-583b185597ad (04ef286e-3020-4895-91ef-583b185597ad): created successfully >2018-08-31 05:17:56.120 288514 INFO cinder.volume.manager [req-f8828022-4d84-4f39-826f-d519b3eb1345 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Created volume successfully. >2018-08-31 05:18:00.144 288514 INFO cinder.volume.manager [req-921f3873-9cb6-4bd7-bf64-cab1ff8d584a f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Create snapshot completed successfully >2018-08-31 05:18:00.626 288514 INFO cinder.volume.flows.manager.create_volume [req-41dd17a8-1cc0-4a0c-b7c1-157d2ca762ff f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Volume ad09fc93-001d-4669-8d38-0431bdd996ba: being created as snap with specification: {'status': u'creating', 'volume_size': 2, 'snapshot_id': '26d78d24-9229-46f8-b555-297fbc749538', 'volume_name': 'volume-ad09fc93-001d-4669-8d38-0431bdd996ba'} >2018-08-31 05:19:02.395 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-41dd17a8-1cc0-4a0c-b7c1-157d2ca762ff f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Unable to get remote copy information for volume volume-ad09fc93-001d-4669-8d38-0431bdd996ba >2018-08-31 05:19:03.979 288514 INFO cinder.volume.flows.manager.create_volume [req-41dd17a8-1cc0-4a0c-b7c1-157d2ca762ff f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Volume volume-ad09fc93-001d-4669-8d38-0431bdd996ba (ad09fc93-001d-4669-8d38-0431bdd996ba): created successfully >2018-08-31 05:19:03.984 288514 INFO cinder.volume.manager [req-41dd17a8-1cc0-4a0c-b7c1-157d2ca762ff f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Created volume successfully. >2018-08-31 05:19:07.213 288514 INFO cinder.volume.manager [req-2703bfd6-55d5-49aa-825e-c5bac269e8a4 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Deleted volume successfully. >2018-08-31 05:19:12.139 288753 INFO cinder.volume.manager [req-ba8a0ad7-71dc-4147-a9c5-4149308409b2 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:19:13.475 288514 INFO cinder.volume.manager [req-b4992b73-2275-404e-993e-c455dac37175 f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Delete snapshot completed successfully. >2018-08-31 05:19:17.481 288514 WARNING cinder.db.sqlalchemy.api [req-7b995d43-74e0-4b4c-b041-d94d10f9c0da f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Change will make usage less than 0 for the following resources: ['gigabytes', 'volumes'] >2018-08-31 05:19:17.502 288753 INFO cinder.volume.manager [req-4fa1dbec-b084-489e-9154-9d0cab87aaea f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Deleted volume successfully. >2018-08-31 05:19:17.758 288514 INFO cinder.volume.manager [req-7b995d43-74e0-4b4c-b041-d94d10f9c0da f18fed4140924a5e8d044443c7da0cff 95119a563b5e4853ad7c73e456d985ce - default default] Deleted volume successfully. >2018-08-31 05:19:22.118 288514 INFO cinder.volume.flows.manager.create_volume [req-ca5a63ec-8341-4a0d-88ce-7626fdf2bb1a 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Volume 7a3ae857-ebd7-4606-b70c-36f9fbb32926: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-7a3ae857-ebd7-4606-b70c-36f9fbb32926'} >2018-08-31 05:19:23.392 288514 INFO cinder.volume.flows.manager.create_volume [req-ca5a63ec-8341-4a0d-88ce-7626fdf2bb1a 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Volume volume-7a3ae857-ebd7-4606-b70c-36f9fbb32926 (7a3ae857-ebd7-4606-b70c-36f9fbb32926): created successfully >2018-08-31 05:19:23.396 288514 INFO cinder.volume.manager [req-ca5a63ec-8341-4a0d-88ce-7626fdf2bb1a 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Created volume successfully. >2018-08-31 05:19:26.530 288514 INFO cinder.volume.manager [req-2c577e11-2036-4215-9f39-19b89b7da1c8 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Create snapshot completed successfully >2018-08-31 05:19:29.464 288514 INFO cinder.volume.manager [req-4f412130-37ed-443e-b90b-7e9be0c401ac 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Create snapshot completed successfully >2018-08-31 05:19:32.200 288514 INFO cinder.volume.manager [req-ca3dae53-9dfb-4a33-b093-d5ef66f0b99e 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Create snapshot completed successfully >2018-08-31 05:19:39.714 288514 INFO cinder.volume.manager [req-515532e4-a86a-472e-9fbd-db01c81e90a3 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Delete snapshot completed successfully. >2018-08-31 05:19:40.735 288514 INFO cinder.volume.manager [req-c371f3b1-cec3-4e70-a8ff-f6958a14c7ae 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Delete snapshot completed successfully. >2018-08-31 05:19:41.746 288514 INFO cinder.volume.manager [req-a94588f8-e9e6-4be2-ad8f-99c3e1943177 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Delete snapshot completed successfully. >2018-08-31 05:19:43.869 288514 INFO cinder.volume.manager [req-30519c21-5305-4f48-b693-7dbaa2084de6 5d9b7af6474f42a59617e0c31da01e73 e80f1d0bfcf440ee9fa26149e2a30885 - default default] Deleted volume successfully. >2018-08-31 05:19:49.080 288514 INFO cinder.volume.flows.manager.create_volume [req-01a6a28e-46c1-4e6f-b35d-6262863029ac fa2d4d6b72774514857eeeee3341bbae 29e0442ef5514e728f67134bf2b77cbf - default default] Volume 64ab5158-51d1-45a5-9082-7cd4bbb3d7e8: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-64ab5158-51d1-45a5-9082-7cd4bbb3d7e8'} >2018-08-31 05:19:50.387 288514 INFO cinder.volume.flows.manager.create_volume [req-01a6a28e-46c1-4e6f-b35d-6262863029ac fa2d4d6b72774514857eeeee3341bbae 29e0442ef5514e728f67134bf2b77cbf - default default] Volume volume-64ab5158-51d1-45a5-9082-7cd4bbb3d7e8 (64ab5158-51d1-45a5-9082-7cd4bbb3d7e8): created successfully >2018-08-31 05:19:50.391 288514 INFO cinder.volume.manager [req-01a6a28e-46c1-4e6f-b35d-6262863029ac fa2d4d6b72774514857eeeee3341bbae 29e0442ef5514e728f67134bf2b77cbf - default default] Created volume successfully. >2018-08-31 05:19:54.586 288514 INFO cinder.volume.manager [req-71f162aa-86e2-40b4-996c-3b65bfd96347 fa2d4d6b72774514857eeeee3341bbae 29e0442ef5514e728f67134bf2b77cbf - default default] Create snapshot completed successfully >2018-08-31 05:19:59.203 288514 INFO cinder.volume.manager [req-a6f0a2d4-5c0d-4187-951c-4b121acc4f5c fa2d4d6b72774514857eeeee3341bbae 29e0442ef5514e728f67134bf2b77cbf - default default] Delete snapshot completed successfully. >2018-08-31 05:20:03.119 288514 INFO cinder.volume.manager [req-50f86006-cef5-4de1-b09f-468890d2a72f fa2d4d6b72774514857eeeee3341bbae 29e0442ef5514e728f67134bf2b77cbf - default default] Deleted volume successfully. >2018-08-31 05:20:26.165 288514 INFO cinder.volume.flows.manager.create_volume [req-17ce0b64-526a-4cee-8392-35340a1e319f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume 0bb93495-c635-4981-92eb-1dd05acb43d4: being created as image with specification: {'status': u'creating', 'image_location': (u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', None), 'volume_size': 1, 'volume_name': 'volume-0bb93495-c635-4981-92eb-1dd05acb43d4', 'image_id': '6515695b-edb7-451c-870b-80a7ec9f72db', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f636cd74b10>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'cirros-0.3.5-x86_64-disk.img', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2018, 8, 14, 8, 56, 23, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2018, 8, 14, 8, 56, 26, tzinfo=<iso8601.Utc>), u'visibility': u'public', 'properties': {}, u'owner': u'def303dd8d60408a807bf94835de5e0b', u'protected': False, u'id': u'6515695b-edb7-451c-870b-80a7ec9f72db', u'file': u'/v2/images/6515695b-edb7-451c-870b-80a7ec9f72db/file', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': 0, u'direct_url': u'swift+config://ref1/glance/6515695b-edb7-451c-870b-80a7ec9f72db', u'min_ram': 0, u'size': 13267968}} >2018-08-31 05:20:27.010 288514 INFO cinder.image.image_utils [req-17ce0b64-526a-4cee-8392-35340a1e319f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Image download 12.00 MB at 12.00 MB/s >2018-08-31 05:20:34.143 288514 INFO os_brick.initiator.connectors.iscsi [req-17ce0b64-526a-4cee-8392-35340a1e319f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Trying to connect to iSCSI portal 9.11.54.77:3260 >2018-08-31 05:20:42.112 288514 INFO cinder.image.image_utils [req-17ce0b64-526a-4cee-8392-35340a1e319f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Converted 39.00 MB image at 35.60 MB/s >2018-08-31 05:20:43.879 288514 INFO cinder.volume.flows.manager.create_volume [req-17ce0b64-526a-4cee-8392-35340a1e319f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume volume-0bb93495-c635-4981-92eb-1dd05acb43d4 (0bb93495-c635-4981-92eb-1dd05acb43d4): created successfully >2018-08-31 05:20:43.883 288514 INFO cinder.volume.manager [req-17ce0b64-526a-4cee-8392-35340a1e319f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Created volume successfully. >2018-08-31 05:20:44.440 288514 INFO cinder.volume.flows.manager.create_volume [req-c616660c-1213-4df0-b708-20629228d70f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume 1b551932-5e04-4115-9953-643b13e9a391: being created as source_vol with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-1b551932-5e04-4115-9953-643b13e9a391', 'source_volid': '0bb93495-c635-4981-92eb-1dd05acb43d4', 'source_volstatus': u'available'} >2018-08-31 05:20:46.868 288514 INFO cinder.volume.flows.manager.create_volume [req-c616660c-1213-4df0-b708-20629228d70f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume volume-1b551932-5e04-4115-9953-643b13e9a391 (1b551932-5e04-4115-9953-643b13e9a391): created successfully >2018-08-31 05:20:46.872 288514 INFO cinder.volume.manager [req-c616660c-1213-4df0-b708-20629228d70f 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Created volume successfully. >2018-08-31 05:20:48.414 288514 INFO cinder.volume.flows.manager.create_volume [req-85105388-52ca-46ae-9c30-a5cd2454c643 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume 9c7d7c87-34a7-408c-9c02-60f3e28b119d: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-9c7d7c87-34a7-408c-9c02-60f3e28b119d'} >2018-08-31 05:20:49.690 288514 INFO cinder.volume.flows.manager.create_volume [req-85105388-52ca-46ae-9c30-a5cd2454c643 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume volume-9c7d7c87-34a7-408c-9c02-60f3e28b119d (9c7d7c87-34a7-408c-9c02-60f3e28b119d): created successfully >2018-08-31 05:20:49.694 288514 INFO cinder.volume.manager [req-85105388-52ca-46ae-9c30-a5cd2454c643 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Created volume successfully. >2018-08-31 05:20:51.324 288514 INFO cinder.volume.flows.manager.create_volume [req-7a5a9985-7df0-4e7a-bcc7-d3d4acc31871 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume 61d19350-8626-4740-9f7a-92cbdf182be1: being created as source_vol with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-61d19350-8626-4740-9f7a-92cbdf182be1', 'source_volid': '9c7d7c87-34a7-408c-9c02-60f3e28b119d', 'source_volstatus': u'available'} >2018-08-31 05:21:00.549 288514 INFO cinder.volume.drivers.ibm.storwize_svc.storwize_svc_common [req-7a5a9985-7df0-4e7a-bcc7-d3d4acc31871 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Unable to get remote copy information for volume volume-61d19350-8626-4740-9f7a-92cbdf182be1 >2018-08-31 05:21:01.714 288514 INFO cinder.volume.flows.manager.create_volume [req-7a5a9985-7df0-4e7a-bcc7-d3d4acc31871 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Volume volume-61d19350-8626-4740-9f7a-92cbdf182be1 (61d19350-8626-4740-9f7a-92cbdf182be1): created successfully >2018-08-31 05:21:01.718 288514 INFO cinder.volume.manager [req-7a5a9985-7df0-4e7a-bcc7-d3d4acc31871 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Created volume successfully. >2018-08-31 05:21:03.086 288514 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_7.4) >2018-08-31 05:21:03.172 288514 INFO paramiko.transport [-] Authentication (password) successful! >2018-08-31 05:21:05.988 288514 INFO cinder.volume.manager [req-2686f0dc-d2e2-4858-b4a7-ceb06ebb1008 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Deleted volume successfully. >2018-08-31 05:21:09.102 288514 INFO cinder.volume.manager [req-d8bff652-8c50-4766-b771-a8b400d458d3 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Deleted volume successfully. >2018-08-31 05:21:09.116 288514 INFO cinder.volume.manager [req-8ae81b9c-cea0-4775-9683-dce550aee920 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Deleted volume successfully. >2018-08-31 05:21:09.520 288514 INFO cinder.volume.manager [req-2d6902cb-e47c-4af1-9ef9-d4dc552d85bb 614cf51a844b4bd2ade337a2698d36ff 0b7249425a19472d9cb51d53559fb3c3 - default default] Deleted volume successfully. >2018-08-31 05:21:13.263 288514 INFO cinder.volume.flows.manager.create_volume [req-f38110c3-9a26-46de-91fc-a6ea6bc7a7e3 ec200ea847f14ae5a4355a55fa847e11 c7515793bde64caab09c1ecf3420edf3 - default default] Volume f04a9f9e-37de-490e-a027-86fe5b4bba71: being created as raw with specification: {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-f04a9f9e-37de-490e-a027-86fe5b4bba71'} >2018-08-31 05:21:14.550 288514 INFO cinder.volume.flows.manager.create_volume [req-f38110c3-9a26-46de-91fc-a6ea6bc7a7e3 ec200ea847f14ae5a4355a55fa847e11 c7515793bde64caab09c1ecf3420edf3 - default default] Volume volume-f04a9f9e-37de-490e-a027-86fe5b4bba71 (f04a9f9e-37de-490e-a027-86fe5b4bba71): created successfully >2018-08-31 05:21:14.554 288514 INFO cinder.volume.manager [req-f38110c3-9a26-46de-91fc-a6ea6bc7a7e3 ec200ea847f14ae5a4355a55fa847e11 c7515793bde64caab09c1ecf3420edf3 - default default] Created volume successfully. >2018-08-31 05:21:16.979 288514 INFO cinder.volume.manager [req-8343fd3e-9aca-480e-a94c-1f9a9b0f4f2f ec200ea847f14ae5a4355a55fa847e11 c7515793bde64caab09c1ecf3420edf3 - default default] Deleted volume successfully. >2018-08-31 05:21:41.571 288514 INFO cinder.volume.flows.manager.create_volume [req-72cab249-9501-43f5-977a-da79c0630794 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] Volume 2c55fc36-e635-48d5-ac7d-bb514d9b0f91: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-2c55fc36-e635-48d5-ac7d-bb514d9b0f91'} >2018-08-31 05:21:42.984 288514 INFO cinder.volume.flows.manager.create_volume [req-72cab249-9501-43f5-977a-da79c0630794 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] Volume volume-2c55fc36-e635-48d5-ac7d-bb514d9b0f91 (2c55fc36-e635-48d5-ac7d-bb514d9b0f91): created successfully >2018-08-31 05:21:42.988 288514 INFO cinder.volume.manager [req-72cab249-9501-43f5-977a-da79c0630794 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] Created volume successfully. >2018-08-31 05:21:46.472 288514 INFO cinder.volume.manager [req-de5127e9-3af6-47b3-935f-18c9260380d7 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] Deleted volume successfully. >2018-08-31 05:21:47.874 288514 INFO cinder.volume.flows.manager.create_volume [req-1fd53406-17be-496c-a62e-e940583ee404 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] Volume 15142805-9d6f-41b1-8bf8-00c33af73c20: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-15142805-9d6f-41b1-8bf8-00c33af73c20'} >2018-08-31 05:21:49.282 288514 INFO cinder.volume.flows.manager.create_volume [req-1fd53406-17be-496c-a62e-e940583ee404 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] Volume volume-15142805-9d6f-41b1-8bf8-00c33af73c20 (15142805-9d6f-41b1-8bf8-00c33af73c20): created successfully >2018-08-31 05:21:49.287 288514 INFO cinder.volume.manager [req-1fd53406-17be-496c-a62e-e940583ee404 92364c4c4acb4dc49f3f29daa734375c fa266313fd464e8186fd69e4ccb1bfdc - default default] Created volume successfully. >2018-08-31 05:21:50.883 288514 INFO cinder.volume.manager [req-6588898b-97fb-4fb2-976c-586075eb0148 f855912b0b6a4b3d9f28cfcfdbd3d99c ef07f5b179fa4bd6abb390ef22fa4a5c - default default] Transfer volume completed successfully. >2018-08-31 05:21:53.868 288514 INFO cinder.volume.manager [req-d9d28254-5265-4424-a16a-20bb895bce67 162a0ecf20b342659c3f81b0b3185cb8 2b361968107b40d1b742c1e080e3f8e5 - default default] Deleted volume successfully. >2018-08-31 05:22:02.095 288514 INFO cinder.volume.flows.manager.create_volume [req-d1c6a9ba-29ba-4a14-8d12-3c2325e75c2d d0be689659a34bda82e6394b6d613247 2db8f9ad7a5b44c789c7dbf908ffe3be - default default] Volume eb7a1d80-b466-4834-bb88-83e1bd8cc46f: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-eb7a1d80-b466-4834-bb88-83e1bd8cc46f'} >2018-08-31 05:22:03.378 288514 INFO cinder.volume.flows.manager.create_volume [req-d1c6a9ba-29ba-4a14-8d12-3c2325e75c2d d0be689659a34bda82e6394b6d613247 2db8f9ad7a5b44c789c7dbf908ffe3be - default default] Volume volume-eb7a1d80-b466-4834-bb88-83e1bd8cc46f (eb7a1d80-b466-4834-bb88-83e1bd8cc46f): created successfully >2018-08-31 05:22:03.383 288514 INFO cinder.volume.manager [req-d1c6a9ba-29ba-4a14-8d12-3c2325e75c2d d0be689659a34bda82e6394b6d613247 2db8f9ad7a5b44c789c7dbf908ffe3be - default default] Created volume successfully. >2018-08-31 05:22:08.458 288514 INFO cinder.volume.manager [req-fc380c73-8323-44fc-875f-21c67ad8b972 d0be689659a34bda82e6394b6d613247 2db8f9ad7a5b44c789c7dbf908ffe3be - default default] Deleted volume successfully. >2018-08-31 05:22:14.338 288514 INFO cinder.volume.flows.manager.create_volume [req-429769cc-d0a5-429c-ad54-5f2ec2f5310f d8e1e3087f644c38b8aeb85d41b496cb f58553a51d3a4d61b49ce0d360d2dc13 - default default] Volume 865cda7c-ee04-4d00-a389-2329b1f7b658: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-865cda7c-ee04-4d00-a389-2329b1f7b658'} >2018-08-31 05:22:15.696 288514 INFO cinder.volume.flows.manager.create_volume [req-429769cc-d0a5-429c-ad54-5f2ec2f5310f d8e1e3087f644c38b8aeb85d41b496cb f58553a51d3a4d61b49ce0d360d2dc13 - default default] Volume volume-865cda7c-ee04-4d00-a389-2329b1f7b658 (865cda7c-ee04-4d00-a389-2329b1f7b658): created successfully >2018-08-31 05:22:15.704 288514 INFO cinder.volume.manager [req-429769cc-d0a5-429c-ad54-5f2ec2f5310f d8e1e3087f644c38b8aeb85d41b496cb f58553a51d3a4d61b49ce0d360d2dc13 - default default] Created volume successfully. >2018-08-31 05:22:19.079 288514 INFO cinder.volume.manager [req-e9011b9d-f8aa-4cf8-8406-153f83995386 d8e1e3087f644c38b8aeb85d41b496cb f58553a51d3a4d61b49ce0d360d2dc13 - default default] Create snapshot completed successfully >2018-08-31 05:22:23.829 288514 INFO cinder.volume.manager [req-7415c850-5dd3-4cee-9535-175ef4ea95eb d8e1e3087f644c38b8aeb85d41b496cb f58553a51d3a4d61b49ce0d360d2dc13 - default default] Delete snapshot completed successfully. >2018-08-31 05:22:26.444 288514 INFO cinder.volume.manager [req-cbc7ab1d-f2fc-462b-9f1b-014a3e322cb0 d8e1e3087f644c38b8aeb85d41b496cb f58553a51d3a4d61b49ce0d360d2dc13 - default default] Deleted volume successfully.
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1625223
:
1480764
|
1480765
| 1480768 |
1480769
|
1480770
|
1480771