Bug 1304354 - [ppc64le] Migration failed after hotplug VM memory
Summary: [ppc64le] Migration failed after hotplug VM memory
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt
Version: 3.6.0.3
Hardware: ppc64le
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Michal Skrivanek
QA Contact: Israel Pinto
URL:
Whiteboard:
: 1304364 (view as bug list)
Depends On:
Blocks: RHEV3.6PPC 1284775 1305498
TreeView+ depends on / blocked
 
Reported: 2016-02-03 12:01 UTC by Israel Pinto
Modified: 2016-02-21 11:00 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-03 14:24:48 UTC
oVirt Team: Virt
Embargoed:
rule-engine: planning_ack?
ipinto: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)
engine_log (69.58 KB, application/zip)
2016-02-03 12:02 UTC, Israel Pinto
no flags Details
vdsm_log_host_1 (927.60 KB, application/zip)
2016-02-03 12:03 UTC, Israel Pinto
no flags Details
vdsm_log_host_2 (711.81 KB, application/zip)
2016-02-03 12:03 UTC, Israel Pinto
no flags Details

Description Israel Pinto 2016-02-03 12:01:58 UTC
Description of problem:
After hotplug memory to VM the migration VM failed

Version-Release number of selected component (if applicable):
RHEVM Version: 3.6.3-0.1.el6
vdsm-4.17.19-0.el7ev
libvirt-1.2.17-13.el7_2.3
KVM: 2.3.0 - 31.el7_2.7
kernel: 3.10.0 - 327.10.1.el7.ppc64le

How reproducible:
100%

Steps to Reproduce:
1. Enable hotplug memory via DB
2. Hotplug memory to VM with 1GB
3. Migrate VM

Actual results:
Hotplug memory succeed 
Migration failed

In the engine log i see that it report that there are no host to migration,
this is not true the second host can receive the VM.
Migration with hotplug work fine. 

Expected results:
Migration 

Additional info:
VM name: golden_env_mixed_virtio_1_1
host_1: 10.16.69.249
host_2: 10.16.69.168

vdsm log:
Thread-29688::ERROR::2016-02-03 06:46:21,371::vm::753::virt.vm::(_startUnderlyingVm) vmId=`a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d`::Failed to start a migration destination vm
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 722, in _startUnderlyingVm
    self._completeIncomingMigration()
  File "/usr/share/vdsm/virt/vm.py", line 2852, in _completeIncomingMigration
    self._incomingMigrationFinished.isSet(), usedTimeout)
  File "/usr/share/vdsm/virt/vm.py", line 2911, in _attachLibvirtDomainAfterMigration
    raise MigrationError(e.get_error_message())
MigrationError: Domain not found: no domain with matching uuid 'a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d'
Thread-29688::INFO::2016-02-03 06:46:21,372::vm::1330::virt.vm::(setDownStatus) vmId=`a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d`::Changed state to Down: VM failed to migrate (code=8)
Thread-29688::DEBUG::2016-02-03 06:46:21,373::__init__::206::jsonrpc.Notification::(emit) Sending event {"params": {"notify_time": 43108966330, "a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d": {"status": "Down", "timeOffset": "0", "exitReason": 8, "exitMessage": "VM failed to migrate", "exitCode": 1}}, "jsonrpc": "2.0", "method": "|virt|VM_status|a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d"}

engine log:
2016-02-03 13:46:46,686 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [] Correlation ID: 53a8f6de, Job ID: a867a706-b34e-42a1-a9ae-f6065cec22ab, Call Stack: null, Custom Event ID: -1, Message: Failed to migrate VM golden_env_mixed_virtio_1_1 to Host host_mixed_2 . Trying to migrate to another Host.
2016-02-03 13:46:46,732 WARN  [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-6-thread-50) [] CanDoAction of action 'MigrateVm' failed for user admin@internal. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_NO_HOSTS
2016-02-03 13:46:46,732 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-6-thread-50) [] Lock freed to object 'EngineLock:{exclusiveLocks='[a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_MIGRATED$VmName golden_env_mixed_virtio_1_1>]', sharedLocks='null'}'
2016-02-03 13:46:46,739 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [] Correlation ID: 53a8f6de, Job ID: a867a706-b34e-42a1-a9ae-f6065cec22ab, Call Stack: null, Custom Event ID: -1, Message: No available host was found to migrate VM golden_env_mixed_virtio_1_1 to.
2016-02-03 13:46:46,742 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [] Correlation ID: 53a8f6de, Job ID: a867a706-b34e-42a1-a9ae-f6065cec22ab, Call Stack: null, Custom Event ID: -1, Message: Migration failed  (VM: golden_env_mixed_virtio_1_1, Source: host_mixed_1).

Comment 1 Israel Pinto 2016-02-03 12:02:26 UTC
Created attachment 1120724 [details]
engine_log

Comment 2 Israel Pinto 2016-02-03 12:03:03 UTC
Created attachment 1120725 [details]
vdsm_log_host_1

Comment 3 Israel Pinto 2016-02-03 12:03:41 UTC
Created attachment 1120726 [details]
vdsm_log_host_2

Comment 4 Michal Skrivanek 2016-02-03 14:24:48 UTC
hotplug is not supposed to be enabled (hence resolution is NOTABUG)
but we do want it, of course, and we're tracking that under bug 1284775

Comment 5 Michal Skrivanek 2016-02-03 14:41:36 UTC
and the root cause for migration error is bug 1304300

Comment 6 Michal Skrivanek 2016-02-03 14:42:21 UTC
*** Bug 1304364 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.