Bug 1710040
| Summary: | [RHOS 15] Tempest test failure: test_live_block_migration | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Archit Modi <amodi> | ||||
| Component: | openstack-nova | Assignee: | melanie witt <mwitt> | ||||
| Status: | CLOSED DUPLICATE | QA Contact: | OSP DFG:Compute <osp-dfg-compute> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 15.0 (Stein) | CC: | dasmith, eglynn, jhakimra, kchamart, mbooth, sbauza, sgordon, vromanso | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1710041 (view as bug list) | Environment: | |||||
| Last Closed: | 2019-05-24 14:25:52 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Archit Modi
2019-05-14 19:33:33 UTC
For tempest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration, we see in the nova-compute.log:
2019-05-11 21:19:07.526 8 DEBUG nova.compute.resource_tracker [req-54455960-5c54-4da8-9e43-3aa6d6b40bbd - - - - -] Migration for instance 10097d4c-3bfd-4fd8-8d22-ec322887b683 refers to another host's instance! _pair_instances_to_migrations /usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py:759
2019-05-11 21:19:07.673 8 WARNING nova.compute.resource_tracker [req-54455960-5c54-4da8-9e43-3aa6d6b40bbd - - - - -] Instance 10097d4c-3bfd-4fd8-8d22-ec322887b683 has been moved to another host compute-1.localdomain(compute-1.localdomain). There are allocations remaining against the source host that might need to be removed: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 64, 'VCPU': 1}}.
[1] http://cougar11.scl.lab.tlv.redhat.com/DFG-compute-nova-15_director-rhel-virthost-1cont_2comp-ipv4-geneve-live-migration/6/compute-0.tar.gz?compute-0/var/log/containers/nova/nova-compute.log
For tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_live_block_migration, we see in nova-compute.log:
2019-05-11 21:20:38.051 8 WARNING nova.compute.manager [-] [instance: 8f004f1e-f0d5-438a-8386-40af86c9ec4c] Timed out waiting for events: [('network-vif-plugged', '9b7746a4-a6e7-438d-a394-f91dd8cc2fed')]. If these timeouts are a persistent issue it could mean the networking backend on host compute-1.localdomain does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host compute-1.localdomain.: eventlet.timeout.Timeout: 300 seconds
which shows a timeout waiting for the 'network-vif-plugged' event from neutron.
2019-05-11 21:20:38.141 8 INFO nova.compute.manager [-] [instance: 8f004f1e-f0d5-438a-8386-40af86c9ec4c] Swapping old allocation on 893f3bb8-7710-44a3-bc26-012617217beb held by migration be728c3b-5e43-4e09-965f-e9b4e11abdfa for instance
2019-05-11 21:20:38.232 8 DEBUG nova.scheduler.client.report [-] Overwriting current allocation {'allocations': {'e8eb707f-90d3-4046-8053-a25f468b1269': {'resources': {'DISK_GB': 1, 'MEMORY_MB': 64, 'VCPU': 1}, 'generation': 143}}, 'project_id': '646583384be54cfda42f90b6d2dee13a', 'user_id': '181b6610a53b405a96c5c0fca7bba6fb', 'consumer_generation': 1} on consumer 8f004f1e-f0d5-438a-8386-40af86c9ec4c move_allocations /usr/lib/python3.6/site-packages/nova/scheduler/client/report.py:1883
These ^ log messages are showing the revert of placement allocations, presumably during _rollback_live_migration.
2019-05-11 21:20:45.703 8 DEBUG nova.compute.manager [req-4d54bad3-6491-42d5-a95b-a686e9655791 b2444e993ea64c0fb247340e24e26004 eaf60d0147884211a27211f29f21fc31 - default default] [instance: 8f004f1e-f0d5-438a-8386-40af86c9ec4c] Received event network-changed-9b7746a4-a6e7-438d-a394-f91dd8cc2fed external_instance_event /usr/lib/python3.6/site-packages/nova/compute/manager.py:8357
2019-05-11 21:26:12.377 8 DEBUG nova.compute.manager [req-9fbdc30d-7a73-4bc3-b254-6a727e230746 181b6610a53b405a96c5c0fca7bba6fb 646583384be54cfda42f90b6d2dee13a - default default] [instance: 8f004f1e-f0d5-438a-8386-40af86c9ec4c] Events pending at deletion: network-vif-plugged-9b7746a4-a6e7-438d-a394-f91dd8cc2fed _delete_instance /usr/lib/python3.6/site-packages/nova/compute/manager.py:2723
*** This bug has been marked as a duplicate of bug 1710038 *** |