Bug 1424129
| Summary: | openstack server migrate --live doesn't work | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Marius Cornea <mcornea> | ||||
| Component: | python-openstackclient | Assignee: | Michal Pryc <mpryc> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Archit Modi <amodi> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 10.0 (Newton) | CC: | acanan, aguetta, amodi, anbecker, apevec, brault, ftaylor, gkadam, jjoyce, jpichon, jruzicka, jschluet, lhh, lyarwood, mas-hatada, mbooth, mcornea, mkadmiel, mpryc, mschuppe, rcernin, rhel-osp-bz, rlocke, rlondhe, sferdjao, sknauss, srevivo, svigan, vcojot | ||||
| Target Milestone: | z8 | Keywords: | Triaged, ZStream | ||||
| Target Release: | 10.0 (Newton) | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | python-openstackclient-3.2.1-3.el7ost | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2018-05-17 15:49:28 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Marius Cornea
2017-02-17 16:00:44 UTC
*** Bug 1424066 has been marked as a duplicate of this bug. *** (In reply to Marius Cornea from comment #0) > Description of problem: > openstack server migrate --live fails with the following error: > > [stack@undercloud-0 ~]$ openstack server migrate > st--d0-instance-grw4dxoglahw-my_instance-4cm3la6a6xba --live > overcloud-compute-0.localdomain > overcloud-compute-1.localdomain is not on local storage: Block migration can > not be used with shared storage. (HTTP 400) (Request-ID: > req-6fe46746-c5fc-414a-8bbb-f7e056965396) > > The nova instances are stored on Ceph storage so I expect the live migration > to work. > > nova live-migration st--d0-instance-grw4dxoglahw-my_instance-4cm3la6a6xba > overcloud-compute-0.localdomain works fine instead. > > Version-Release number of selected component (if applicable): > python-openstackclient-3.2.1-1.el7ost.noarch > python-novaclient-6.0.0-1.el7ost.noarch > > How reproducible: > 100% That is little bit confusing, the help message for the command "openstack server migrate" is saying that the default behavior is to use "shared-migration" [0] but the error message returned makes me think a "block-migration" has been initiated. Without any log is difficult to say. So first can you try to execute this same command but specify that you want to use "shared-migration", something like: $ openstack server migrate st--d0-instance-grw4dxoglahw-my_instance-4cm3la6a6xba --shared-migration --live overcloud-compute-0.localdomain [0] https://docs.openstack.org/developer/python-openstackclient/command-objects/server.html#server-migrate Hi Sahid, Please can you or anyone check whether the shared-migration vs block-migration are not switched over? When migrating the instance with local ephemeral disks one seems to must use "--shared-migration" When migrating the instance that is boot from a volume that is shared one seems to must use "--block-migration" Thanks! (In reply to Sahid Ferdjaoui from comment #2) [..] > $ openstack server migrate > st--d0-instance-grw4dxoglahw-my_instance-4cm3la6a6xba --shared-migration > --live overcloud-compute-0.localdomain It has the same result: [stack@undercloud-0 ~]$ openstack server migrate st--b2-instance-2czjr4x7peym-my_instance-rhppwfsbddpd --shared-migration --live overcloud-compute-1.localdomain overcloud-compute-0.localdomain is not on local storage: Block migration can not be used with shared storage. (HTTP 400) (Request-ID: req-40cc689c-66d4-4640-84da-128e0d9b869b) > [0] > https://docs.openstack.org/developer/python-openstackclient/command-objects/ > server.html#server-migrate Created attachment 1272239 [details]
nova-conductor.log
Attaching the nova-conductor.log
I'm seeing the same. openstack server migrate --shared-migration --live compute200.cluster 6da6035b-fc96-414e-a0d0-99e057bd90e7 compute232.cluster is not on local storage: Block migration can not be used with shared storage. (HTTP 400) (Request-ID: req-7c9c111b-e9b0-4a05-beff-2638bca81f3f) This seem to be an issue with the openstackclient:
[stack@undercloud-0 ~]$ openstack server show test
+--------------------------------------+----------------------------------------------------------+
| Field | Value |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | compute-1.localdomain |
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1.localdomain |
| OS-EXT-SRV-ATTR:instance_name | instance-00000003 |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2017-07-05T12:31:27.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | private=192.168.0.9 |
| config_drive | |
| created | 2017-07-05T12:31:14Z |
| flavor | m1.small (d8a898c0-5ed1-484f-843b-96558efd7dbf) |
| hostId | 1013a9907516d64716b1db5ccec36a67262b249f24f1173f0bd9eeef |
| id | 11f76706-c034-47c7-a40e-bb223ec7dd2f |
| image | cirros (27ede6d9-96e2-4dd1-9173-8ef566e60a26) |
| key_name | None |
| name | test |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| project_id | 6eeb1132ac7b43d7a97fc2b4eb82d461 |
| properties | |
| security_groups | [{u'name': u'default'}] |
| status | ACTIVE |
| updated | 2017-07-05T12:34:11Z |
| user_id | 7fab49be19df4b499eeaf6bbd67bc93c |
+--------------------------------------+----------------------------------------------------------+
[stack@undercloud-0 ~]$ openstack server migrate test --live compute-0.localdomain
compute-1.localdomain is not on local storage: Block migration can not be used with shared storage. (HTTP 400) (Request-ID: req-e1713e69-e05f-4e4f-8582-f7dc64210a62)
From nova API log:
2017-07-05 12:36:40.323 5178 DEBUG nova.api.openstack.wsgi [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Action: 'action', calling method: <bound method MigrateServerController._migrate_live of <nova.api.openstack.compute.migrate_server.MigrateServerController object at 0x8d272d0>>, body: {"os-migrateLive": {"disk_over_commit": false, "block_migration": true, "host": "compute-0.localdomain"}} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:633
2017-07-05 12:36:40.325 5178 DEBUG nova.compute.api [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] [instance: 11f76706-c034-47c7-a40e-bb223ec7dd2f] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2213
2017-07-05 12:36:40.400 5178 DEBUG nova.compute.api [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] [instance: 11f76706-c034-47c7-a40e-bb223ec7dd2f] Going to try to live migrate instance to compute-0.localdomain live_migrate /usr/lib/python2.7/site-packages/nova/compute/api.py:3621
2017-07-05 12:36:40.480 5178 DEBUG nova.objects.instance [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Lazy-loading 'flavor' on Instance uuid 11f76706-c034-47c7-a40e-bb223ec7dd2f obj_load_attr /usr/lib/python2.7/site-packages/nova/objects/instance.py:1013
2017-07-05 12:36:40.557 5178 WARNING oslo_config.cfg [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Option "rabbit_hosts" from group "oslo_messaging_rabbit" is deprecated for removal. Its value may be silently ignored in the future.
2017-07-05 12:36:40.557 5178 WARNING oslo_config.cfg [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal. Its value may be silently ignored in the future.
2017-07-05 12:36:40.558 5178 WARNING oslo_config.cfg [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal. Its value may be silently ignored in the future.
2017-07-05 12:36:40.577 5178 DEBUG oslo_messaging._drivers.amqpdriver [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CAST unique_id: 626c1a469b164b7e857cf01e82132fbd NOTIFY exchange 'nova' topic 'notifications.info' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:479
2017-07-05 12:36:40.589 5178 DEBUG oslo_messaging._drivers.amqpdriver [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CAST unique_id: 360181b07a604cf5a510aae14ade40ec NOTIFY exchange 'nova' topic 'versioned_notifications.info' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:479
2017-07-05 12:36:40.685 5178 DEBUG oslo_messaging._drivers.amqpdriver [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CALL msg_id: 1106eb63241141468719d706b2b13852 exchange 'nova' topic 'conductor' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:495
2017-07-05 12:36:41.486 5178 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 1106eb63241141468719d706b2b13852 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:343
2017-07-05 12:36:41.487 5178 INFO nova.api.openstack.wsgi [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] HTTP exception thrown: compute-1.localdomain is not on local storage: Block migration can not be used with shared storage.
2017-07-05 12:36:41.488 5178 DEBUG nova.api.openstack.wsgi [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Returning 400 to user: compute-1.localdomain is not on local storage: Block migration can not be used with shared storage. __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1044
2017-07-05 12:36:41.489 5178 INFO nova.osapi_compute.wsgi.server [req-e1713e69-e05f-4e4f-8582-f7dc64210a62 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] 172.17.1.19 "POST /v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f/action HTTP/1.1" status: 400 len: 476 time: 1.1738911
-> we see '"block_migration": true' even the default is false
Also when we explicitly specify shared-migration we have the '"block_migration": true'
[stack@undercloud-0 ~]$ openstack server migrate test --shared-migration --live compute-0.localdomain --debug
START with options: [u'server', u'migrate', u'test', u'--shared-migration', u'--live', u'compute-0.localdomain', u'--debug']
options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', aodh_endpoint='', auth_type='', auth_url='http://10.0.0.106:5000/v2.0', authorization_code='', cacert=None, cert='', client_id='', client_secret='***', cloud='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, inspector_api_version='1', inspector_url=None, interface='', key='', log_file=None, murano_url='', old_profile=None, openid_scope='', os_alarming_api_version='2', os_application_catalog_api_version='1', os_baremetal_api_version='1.6', os_beta_command=False, os_compute_api_version='', os_container_infra_api_version='1', os_data_processing_api_version='1.1', os_data_processing_url='', os_dns_api_version='2', os_identity_api_version='', os_image_api_version='', os_key_manager_api_version='1', os_metrics_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_queues_api_version='2', os_tripleoclient_api_version='1', os_volume_api_version='', os_workflow_api_version='2', passcode='', password='***', profile=None, project_domain_id='', project_domain_name='', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='', roles='', timing=False, token='***', trust_id='', url='', user='', user_domain_id='', user_domain_name='', user_id='', username='admin', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, 'tripleoclient_api_version': '1', u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'inspector_api_version': '1', 'auth_url': 'http://10.0.0.106:5000/v2.0', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'admin', 'container_infra_api_version': '1', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': '1.6', 'queues_api_version': '2', 'auth': {'project_name': 'admin'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'application_catalog_api_version': '1', 'cacert': None, u'key_manager_api_version': '1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': u'2.0', 'workflow_api_version': '2', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', u'container_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'cacert': None, u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'key_manager_api_version': u'v1', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', 'beta_command': False, 'tripleoclient_api_version': '1', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'inspector_api_version': '1', 'auth_url': 'http://10.0.0.106:5000/v2.0', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'admin', 'container_infra_api_version': '1', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': '1.6', 'queues_api_version': '2', 'auth': {'username': 'admin', 'project_name': 'admin', 'password': '***', 'auth_url': 'http://10.0.0.106:5000/v2.0'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': '1', 'timing': False, 'password': '***', 'application_catalog_api_version': '1', 'cacert': None, u'key_manager_api_version': '1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': u'2.0', 'workflow_api_version': '2', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
compute API version 2, cmd group openstack.compute.v2
network API version 2, cmd group openstack.network.v2
image API version 2, cmd group openstack.image.v2
volume API version 2, cmd group openstack.volume.v2
identity API version 2.0, cmd group openstack.identity.v2
object_store API version 1, cmd group openstack.object_store.v1
neutronclient API version 2, cmd group openstack.neutronclient.v2
messaging API version 2, cmd group openstack.messaging.v2
orchestration API version 1, cmd group openstack.orchestration.v1
workflow_engine API version 2, cmd group openstack.workflow_engine.v2
baremetal API version 1.6, cmd group openstack.baremetal.v1
baremetal_introspection API version 1, cmd group openstack.baremetal_introspection.v1
tripleoclient API version 1, cmd group openstack.tripleoclient.v1
application_catalog API version 1, cmd group openstack.application_catalog.v1
key_manager API version 1, cmd group openstack.key_manager.v1
container_infra API version 1, cmd group openstack.container_infra.v1
dns API version 2, cmd group openstack.dns.v2
alarming API version 2, cmd group openstack.alarming.v2
metric API version 1, cmd group openstack.metric.v1
data_processing API version 1.1, cmd group openstack.data_processing.v1
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, 'tripleoclient_api_version': '1', u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'inspector_api_version': '1', 'auth_url': 'http://10.0.0.106:5000/v2.0', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'admin', 'container_infra_api_version': '1', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': '1.6', 'queues_api_version': '2', 'auth': {'project_name': 'admin'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'application_catalog_api_version': '1', 'cacert': None, u'key_manager_api_version': '1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': u'2.0', 'workflow_api_version': '2', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', u'container_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, 'tripleoclient_api_version': '1', u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', 'metrics_api_version': '1', 'data_processing_api_version': '1.1', 'inspector_api_version': '1', 'auth_url': 'http://10.0.0.106:5000/v2.0', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'admin', 'container_infra_api_version': '1', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': '1.6', 'queues_api_version': '2', 'auth': {'project_name': 'admin'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'application_catalog_api_version': '1', 'cacert': None, u'key_manager_api_version': '1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': u'2.0', 'workflow_api_version': '2', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', u'container_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
command: server migrate -> openstackclient.compute.v2.server.MigrateServer
Using auth plugin: password
Using parameters {'username': 'admin', 'project_name': 'admin', 'password': '***', 'auth_url': 'http://10.0.0.106:5000/v2.0'}
Get auth_ref
REQ: curl -g -i -X GET http://10.0.0.106:5000/v2.0 -H "Accept: application/json" -H "User-Agent: osc-lib keystoneauth1/2.12.3 python-requests/2.10.0 CPython/2.7.5"
Starting new HTTP connection (1): 10.0.0.106
"GET /v2.0 HTTP/1.1" 200 227
RESP: [200] Date: Wed, 05 Jul 2017 12:47:55 GMT Server: Apache Vary: X-Auth-Token,Accept-Encoding x-openstack-request-id: req-bd670615-a76c-4fc2-b271-fec659b8e1b4 Content-Encoding: gzip Content-Length: 227 Content-Type: application/json
RESP BODY: {"version": {"status": "deprecated", "updated": "2016-08-04T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.0.0.106:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}
Making authentication request to http://10.0.0.106:5000/v2.0/tokens
"POST /v2.0/tokens HTTP/1.1" 200 1087
run(Namespace(disk_overcommit=False, live=u'compute-0.localdomain', server=u'test', shared_migration=True, wait=False))
Instantiating compute client for API Version Major: 2, Minor: 0
Making authentication request to http://10.0.0.106:5000/v2.0/tokens
"POST /v2.0/tokens HTTP/1.1" 200 1085
REQ: curl -g -i -X GET http://10.0.0.106:8774/v2.1/servers/test -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}be7810a2c50fc9b2baba8f2ad3b16b8359876f63"
Starting new HTTP connection (1): 10.0.0.106
"GET /v2.1/servers/test HTTP/1.1" 404 79
RESP: [404] Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 79 X-Compute-Request-Id: req-7ef45fe8-6622-4780-9d78-85202c597e33 Date: Wed, 05 Jul 2017 12:47:56 GMT
RESP BODY: {"itemNotFound": {"message": "Instance test could not be found.", "code": 404}}
GET call to compute for http://10.0.0.106:8774/v2.1/servers/test used request id req-7ef45fe8-6622-4780-9d78-85202c597e33
REQ: curl -g -i -X GET http://10.0.0.106:8774/v2.1/servers/test -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}be7810a2c50fc9b2baba8f2ad3b16b8359876f63"
"GET /v2.1/servers/test HTTP/1.1" 404 79
RESP: [404] Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 79 X-Compute-Request-Id: req-7440e916-3a4c-4f7c-9d88-b6eb9b10e90e Date: Wed, 05 Jul 2017 12:47:56 GMT
RESP BODY: {"itemNotFound": {"message": "Instance test could not be found.", "code": 404}}
GET call to compute for http://10.0.0.106:8774/v2.1/servers/test used request id req-7440e916-3a4c-4f7c-9d88-b6eb9b10e90e
REQ: curl -g -i -X GET http://10.0.0.106:8774/v2.1/servers?name=test -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}be7810a2c50fc9b2baba8f2ad3b16b8359876f63"
"GET /v2.1/servers?name=test HTTP/1.1" 200 289
RESP: [200] Content-Length: 289 Content-Type: application/json Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-f9cd6b2e-a9b0-405c-a4c2-4c90b0d073ca Date: Wed, 05 Jul 2017 12:47:56 GMT
RESP BODY: {"servers": [{"id": "11f76706-c034-47c7-a40e-bb223ec7dd2f", "links": [{"href": "http://10.0.0.106:8774/v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f", "rel": "self"}, {"href": "http://10.0.0.106:8774/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f", "rel": "bookmark"}], "name": "test"}]}
GET call to compute for http://10.0.0.106:8774/v2.1/servers?name=test used request id req-f9cd6b2e-a9b0-405c-a4c2-4c90b0d073ca
REQ: curl -g -i -X GET http://10.0.0.106:8774/v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}be7810a2c50fc9b2baba8f2ad3b16b8359876f63"
"GET /v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f HTTP/1.1" 200 1632
RESP: [200] Content-Length: 1632 Content-Type: application/json Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-fd2515ae-5107-4934-8be4-7b9af23a6850 Date: Wed, 05 Jul 2017 12:47:57 GMT
RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {"private": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:b4:b4:96", "version": 4, "addr": "192.168.0.9", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": "http://10.0.0.106:8774/v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f", "rel": "self"}, {"href": "http://10.0.0.106:8774/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f", "rel": "bookmark"}], "image": {"id": "27ede6d9-96e2-4dd1-9173-8ef566e60a26", "links": [{"href": "http://10.0.0.106:8774/images/27ede6d9-96e2-4dd1-9173-8ef566e60a26", "rel": "bookmark"}]}, "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-00000003", "OS-SRV-USG:launched_at": "2017-07-05T12:31:27.000000", "flavor": {"id": "d8a898c0-5ed1-484f-843b-96558efd7dbf", "links": [{"href": "http://10.0.0.106:8774/flavors/d8a898c0-5ed1-484f-843b-96558efd7dbf", "rel": "bookmark"}]}, "id": "11f76706-c034-47c7-a40e-bb223ec7dd2f", "security_groups": [{"name": "default"}], "user_id": "7fab49be19df4b499eeaf6bbd67bc93c", "OS-DCF:diskConfig": "MANUAL", "accessIPv4": "", "accessIPv6": "", "progress": 0, "OS-EXT-STS:power_state": 1, "OS-EXT-AZ:availability_zone": "nova", "metadata": {}, "status": "ACTIVE", "updated": "2017-07-05T12:45:18Z", "hostId": "1013a9907516d64716b1db5ccec36a67262b249f24f1173f0bd9eeef", "OS-EXT-SRV-ATTR:host": "compute-1.localdomain", "OS-SRV-USG:terminated_at": null, "key_name": null, "OS-EXT-SRV-ATTR:hypervisor_hostname": "compute-1.localdomain", "name": "test", "created": "2017-07-05T12:31:14Z", "tenant_id": "6eeb1132ac7b43d7a97fc2b4eb82d461", "os-extended-volumes:volumes_attached": [], "config_drive": ""}}
GET call to compute for http://10.0.0.106:8774/v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f used request id req-fd2515ae-5107-4934-8be4-7b9af23a6850
REQ: curl -g -i -X POST http://10.0.0.106:8774/v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f/action -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}be7810a2c50fc9b2baba8f2ad3b16b8359876f63" -d '{"os-migrateLive": {"disk_over_commit": false, "block_migration": true, "host": "compute-0.localdomain"}}'
"POST /v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f/action HTTP/1.1" 400 143
RESP: [400] Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 143 X-Compute-Request-Id: req-1c9382d5-0495-431c-a9b6-56af2a027a0a Date: Wed, 05 Jul 2017 12:47:58 GMT---
RESP BODY: {"badRequest": {"message": "compute-1.localdomain is not on local storage: Block migration can not be used with shared storage.", "code": 400}}
POST call to compute for http://10.0.0.106:8774/v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f/action used request id req-1c9382d5-0495-431c-a9b6-56af2a027a0a
compute-1.localdomain is not on local storage: Block migration can not be used with shared storage. (HTTP 400) (Request-ID: req-1c9382d5-0495-431c-a9b6-56af2a027a0a)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 387, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
return super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/site-packages/cliff/command.py", line 59, in run
return self.take_action(parsed_args) or 0
File "/usr/lib/python2.7/site-packages/openstackclient/compute/v2/server.py", line 1021, in take_action
parsed_args.disk_overcommit,
File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 402, in substitution
return methods[-1].func(obj, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 433, in live_migrate
disk_over_commit)---
File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 402, in substitution
return methods[-1].func(obj, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1635, in live_migrate
'disk_over_commit': disk_over_commit})
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1845, in _action
info=info, **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1856, in _action_return_resp_and_body
return self.api.client.post(url, body=body)
File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 193, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 178, in request
raise exceptions.from_response(resp, body, url, method)
BadRequest: compute-1.localdomain is not on local storage: Block migration can not be used with shared storage. (HTTP 400) (Request-ID: req-1c9382d5-0495-431c-a9b6-56af2a027a0a)
clean_up MigrateServer: compute-1.localdomain is not on local storage: Block migration can not be used with shared storage. (HTTP 400) (Request-ID: req-1c9382d5-0495-431c-a9b6-56af2a027a0a)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 135, in run
ret_val = super(OpenStackShell, self).run(argv)
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 267, in run
result = self.run_subcommand(remainder)
File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 180, in run_subcommand
ret_value = super(OpenStackShell, self).run_subcommand(argv)
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 387, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
return super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/site-packages/cliff/command.py", line 59, in run
return self.take_action(parsed_args) or 0
File "/usr/lib/python2.7/site-packages/openstackclient/compute/v2/server.py", line 1021, in take_action
parsed_args.disk_overcommit,
File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 402, in substitution
return methods[-1].func(obj, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 433, in live_migrate
disk_over_commit)---
File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 402, in substitution
return methods[-1].func(obj, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1635, in live_migrate
'disk_over_commit': disk_over_commit})
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1845, in _action
info=info, **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1856, in _action_return_resp_and_body
return self.api.client.post(url, body=body)
File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 193, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 178, in request
raise exceptions.from_response(resp, body, url, method)
BadRequest: compute-1.localdomain is not on local storage: Block migration can not be used with shared storage. (HTTP 400) (Request-ID: req-1c9382d5-0495-431c-a9b6-56af2a027a0a)
END return value: 1
2017-07-05 12:47:57.255 5177 DEBUG nova.api.openstack.wsgi [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Action: 'action', calling method: <bound method MigrateServerController._migrate_live of <nova.api.openstack.compute.migrate_server.MigrateServerController object at 0x8d272d0>>, body: {"os-migrateLive": {"disk_over_commit": false, "block_migration": true, "host": "compute-0.localdomain"}} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:633
2017-07-05 12:47:57.256 5177 DEBUG nova.compute.api [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] [instance: 11f76706-c034-47c7-a40e-bb223ec7dd2f] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2213
2017-07-05 12:47:57.336 5177 DEBUG nova.compute.api [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] [instance: 11f76706-c034-47c7-a40e-bb223ec7dd2f] Going to try to live migrate instance to compute-0.localdomain live_migrate /usr/lib/python2.7/site-packages/nova/compute/api.py:3621
2017-07-05 12:47:57.417 5177 DEBUG nova.objects.instance [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Lazy-loading 'flavor' on Instance uuid 11f76706-c034-47c7-a40e-bb223ec7dd2f obj_load_attr /usr/lib/python2.7/site-packages/nova/objects/instance.py:1013
2017-07-05 12:47:57.684 5177 DEBUG oslo_messaging._drivers.amqpdriver [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CAST unique_id: 4ec96e79baa745cc92359351d2b45f0e NOTIFY exchange 'nova' topic 'notifications.info' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:479
2017-07-05 12:47:57.691 5177 DEBUG oslo_messaging._drivers.amqpdriver [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CAST unique_id: 79a939ad16344ce6a46a36b59ceeec85 NOTIFY exchange 'nova' topic 'versioned_notifications.info' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:479
2017-07-05 12:47:57.727 5177 DEBUG oslo_messaging._drivers.amqpdriver [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CALL msg_id: 4566a1e4c05f4fad805a631f62c25159 exchange 'nova' topic 'conductor' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:495
2017-07-05 12:47:58.486 5177 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 4566a1e4c05f4fad805a631f62c25159 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:343
2017-07-05 12:47:58.489 5177 INFO nova.api.openstack.wsgi [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] HTTP exception thrown: compute-1.localdomain is not on local storage: Block migration can not be used with shared storage.
2017-07-05 12:47:58.490 5177 DEBUG nova.api.openstack.wsgi [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Returning 400 to user: compute-1.localdomain is not on local storage: Block migration can not be used with shared storage. __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1044
2017-07-05 12:47:58.492 5177 INFO nova.osapi_compute.wsgi.server [req-1c9382d5-0495-431c-a9b6-56af2a027a0a 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] 172.17.1.19 "POST /v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f/action HTTP/1.1" status: 400 len: 476 time: 1.2447522
Note - the same works with the novaclient:
[stack@undercloud-0 ~]$ nova live-migration test compute-1.localdomain
[stack@undercloud-0 ~]$ openstack server show test
+--------------------------------------+----------------------------------------------------------+
| Field | Value |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | compute-1.localdomain |
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1.localdomain |
| OS-EXT-SRV-ATTR:instance_name | instance-00000003 |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2017-07-05T12:31:27.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | private=192.168.0.9 |
| config_drive | |
| created | 2017-07-05T12:31:14Z |
| flavor | m1.small (d8a898c0-5ed1-484f-843b-96558efd7dbf) |
| hostId | cb3102621af3618ae1cb7b0c97eeadea9010da9584efe910c4afd58c |
| id | 11f76706-c034-47c7-a40e-bb223ec7dd2f |
| image | cirros (27ede6d9-96e2-4dd1-9173-8ef566e60a26) |
| key_name | None |
| name | test |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| project_id | 6eeb1132ac7b43d7a97fc2b4eb82d461 |
| properties | |
| security_groups | [{u'name': u'default'}] |
| status | ACTIVE |
| updated | 2017-07-05T13:01:18Z |
| user_id | 7fab49be19df4b499eeaf6bbd67bc93c |
+--------------------------------------+----------------------------------------------------------+
2017-07-05 13:29:40.404 5176 DEBUG nova.api.openstack.wsgi [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Action: 'action', calling method: <bound method MigrateServerController._migrate_live of <nova.api.openstack.compute.migrate_server.MigrateServerController object at 0x8d272d0>>, body: {"os-migrateLive": {"block_migration": "auto", "host": "compute-1.localdomain"}} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:633
2017-07-05 13:29:40.405 5176 DEBUG nova.compute.api [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] [instance: 11f76706-c034-47c7-a40e-bb223ec7dd2f] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2213
2017-07-05 13:29:40.488 5176 DEBUG nova.compute.api [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] [instance: 11f76706-c034-47c7-a40e-bb223ec7dd2f] Going to try to live migrate instance to compute-1.localdomain live_migrate /usr/lib/python2.7/site-packages/nova/compute/api.py:3621
2017-07-05 13:29:40.578 5176 DEBUG nova.objects.instance [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] Lazy-loading 'flavor' on Instance uuid 11f76706-c034-47c7-a40e-bb223ec7dd2f obj_load_attr /usr/lib/python2.7/site-packages/nova/objects/instance.py:1013
2017-07-05 13:29:40.660 5176 DEBUG oslo_messaging._drivers.amqpdriver [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CAST unique_id: 364bf199782840929fb581bd7f1e2755 NOTIFY exchange 'nova' topic 'notifications.info' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:479
2017-07-05 13:29:40.668 5176 DEBUG oslo_messaging._drivers.amqpdriver [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CAST unique_id: 1657e4f62bc94b25aedded24bffe9bb5 NOTIFY exchange 'nova' topic 'versioned_notifications.info' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:479
2017-07-05 13:29:40.708 5176 DEBUG oslo_messaging._drivers.amqpdriver [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] CAST unique_id: 1bda400af08d46d19ae3888f03560533 exchange 'nova' topic 'conductor' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:495
2017-07-05 13:29:40.713 5176 INFO nova.osapi_compute.wsgi.server [req-8da9379b-29db-445a-b58a-9c01b6bf4396 7fab49be19df4b499eeaf6bbd67bc93c 6eeb1132ac7b43d7a97fc2b4eb82d461 - default default] 172.17.1.19 "POST /v2.1/servers/11f76706-c034-47c7-a40e-bb223ec7dd2f/action HTTP/1.1" status: 202 len: 315 time: 0.3172231
here we see '"block_migration": "auto"'
Fixed upstream with https://review.openstack.org/#/c/385138/ . After applying that patch live migration with shared storage (tested with ceph) works. Nice debugging Martin. Have cherry-picked change upstream, if it helps. Backport has landed upstream so perhaps a Red Hatter can pull this into RHOSP please? Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1596 |