Bug 1304354 - [ppc64le] Migration failed after hotplug VM memory
[ppc64le] Migration failed after hotplug VM memory
Status: CLOSED NOTABUG
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt (Show other bugs)
3.6.0.3
ppc64le Unspecified
unspecified Severity high (vote)
: ---
: ---
Assigned To: Michal Skrivanek
Israel Pinto
:
: 1304364 (view as bug list)
Depends On:
Blocks: 1284775 RHEV3.6PPC 1305498
  Show dependency treegraph
 
Reported: 2016-02-03 07:01 EST by Israel Pinto
Modified: 2016-02-21 06:00 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-03 09:24:48 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: planning_ack?
ipinto: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)
engine_log (69.58 KB, application/zip)
2016-02-03 07:02 EST, Israel Pinto
no flags Details
vdsm_log_host_1 (927.60 KB, application/zip)
2016-02-03 07:03 EST, Israel Pinto
no flags Details
vdsm_log_host_2 (711.81 KB, application/zip)
2016-02-03 07:03 EST, Israel Pinto
no flags Details

  None (edit)
Description Israel Pinto 2016-02-03 07:01:58 EST
Description of problem:
After hotplug memory to VM the migration VM failed

Version-Release number of selected component (if applicable):
RHEVM Version: 3.6.3-0.1.el6
vdsm-4.17.19-0.el7ev
libvirt-1.2.17-13.el7_2.3
KVM: 2.3.0 - 31.el7_2.7
kernel: 3.10.0 - 327.10.1.el7.ppc64le

How reproducible:
100%

Steps to Reproduce:
1. Enable hotplug memory via DB
2. Hotplug memory to VM with 1GB
3. Migrate VM

Actual results:
Hotplug memory succeed 
Migration failed

In the engine log i see that it report that there are no host to migration,
this is not true the second host can receive the VM.
Migration with hotplug work fine. 

Expected results:
Migration 

Additional info:
VM name: golden_env_mixed_virtio_1_1
host_1: 10.16.69.249
host_2: 10.16.69.168

vdsm log:
Thread-29688::ERROR::2016-02-03 06:46:21,371::vm::753::virt.vm::(_startUnderlyingVm) vmId=`a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d`::Failed to start a migration destination vm
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 722, in _startUnderlyingVm
    self._completeIncomingMigration()
  File "/usr/share/vdsm/virt/vm.py", line 2852, in _completeIncomingMigration
    self._incomingMigrationFinished.isSet(), usedTimeout)
  File "/usr/share/vdsm/virt/vm.py", line 2911, in _attachLibvirtDomainAfterMigration
    raise MigrationError(e.get_error_message())
MigrationError: Domain not found: no domain with matching uuid 'a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d'
Thread-29688::INFO::2016-02-03 06:46:21,372::vm::1330::virt.vm::(setDownStatus) vmId=`a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d`::Changed state to Down: VM failed to migrate (code=8)
Thread-29688::DEBUG::2016-02-03 06:46:21,373::__init__::206::jsonrpc.Notification::(emit) Sending event {"params": {"notify_time": 43108966330, "a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d": {"status": "Down", "timeOffset": "0", "exitReason": 8, "exitMessage": "VM failed to migrate", "exitCode": 1}}, "jsonrpc": "2.0", "method": "|virt|VM_status|a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d"}

engine log:
2016-02-03 13:46:46,686 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [] Correlation ID: 53a8f6de, Job ID: a867a706-b34e-42a1-a9ae-f6065cec22ab, Call Stack: null, Custom Event ID: -1, Message: Failed to migrate VM golden_env_mixed_virtio_1_1 to Host host_mixed_2 . Trying to migrate to another Host.
2016-02-03 13:46:46,732 WARN  [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-6-thread-50) [] CanDoAction of action 'MigrateVm' failed for user admin@internal. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_NO_HOSTS
2016-02-03 13:46:46,732 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-6-thread-50) [] Lock freed to object 'EngineLock:{exclusiveLocks='[a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_MIGRATED$VmName golden_env_mixed_virtio_1_1>]', sharedLocks='null'}'
2016-02-03 13:46:46,739 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [] Correlation ID: 53a8f6de, Job ID: a867a706-b34e-42a1-a9ae-f6065cec22ab, Call Stack: null, Custom Event ID: -1, Message: No available host was found to migrate VM golden_env_mixed_virtio_1_1 to.
2016-02-03 13:46:46,742 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [] Correlation ID: 53a8f6de, Job ID: a867a706-b34e-42a1-a9ae-f6065cec22ab, Call Stack: null, Custom Event ID: -1, Message: Migration failed  (VM: golden_env_mixed_virtio_1_1, Source: host_mixed_1).
Comment 1 Israel Pinto 2016-02-03 07:02 EST
Created attachment 1120724 [details]
engine_log
Comment 2 Israel Pinto 2016-02-03 07:03 EST
Created attachment 1120725 [details]
vdsm_log_host_1
Comment 3 Israel Pinto 2016-02-03 07:03 EST
Created attachment 1120726 [details]
vdsm_log_host_2
Comment 4 Michal Skrivanek 2016-02-03 09:24:48 EST
hotplug is not supposed to be enabled (hence resolution is NOTABUG)
but we do want it, of course, and we're tracking that under bug 1284775
Comment 5 Michal Skrivanek 2016-02-03 09:41:36 EST
and the root cause for migration error is bug 1304300
Comment 6 Michal Skrivanek 2016-02-03 09:42:21 EST
*** Bug 1304364 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.