Bug 1377632

Summary: Provide information in the logs about who and why the VM was migrated automatically by the system
Product: [oVirt] ovirt-engine Reporter: Michael Burman <mburman>
Component: BLL.VirtAssignee: Andrej Krejcir <akrejcir>
Status: CLOSED CURRENTRELEASE QA Contact: Artyom <alukiano>
Severity: high Docs Contact:
Priority: high    
Version: 3.6.7CC: akrejcir, bugs, mburman, mgoldboi, rgolan
Target Milestone: ovirt-4.1.0-alphaKeywords: Triaged
Target Release: 4.1.0.2Flags: rule-engine: ovirt-4.0.z+
rule-engine: ovirt-4.1+
mgoldboi: planning_ack+
rgolan: devel_ack+
mavital: testing_ack+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-01 14:42:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine log
none
debug log none

Description Michael Burman 2016-09-20 08:58:58 UTC
Created attachment 1202781 [details]
engine log

Description of problem:
Provide information in the logs about who and why the VM was migrated automatically by the system.

We facing an issue that happens quite a lot in our automation runs in which the VM is being migrated automatically by the system without any reasonable or visible reason.

In the logs there is no information about who and why the migration started.
We don't have any special migration policy in our clusters, everything is default.

Version-Release number of selected component (if applicable):
4.0.4.3-0.1.el7ev

Comment 1 Yaniv Kaul 2016-09-20 10:29:00 UTC
Even in debug log? (I agree it should not be in debug level, btw).

Comment 2 Michael Burman 2016-09-20 11:05:21 UTC
Even in debug mode.

Comment 3 Michael Burman 2016-09-20 11:05:46 UTC
Created attachment 1202853 [details]
debug log

Comment 4 Roy Golan 2016-09-21 06:41:49 UTC
We are missing the migration reason and that should be added. The specific places we use non-user migration when:
- Load balancing
- VM affinity enforcing
- Host Maintenance, which triggers migration internally

A special case where engine doesn't trigger the migration is the when HE agent is migrating the VM when the agent is put to local maintenance. The engine does have an API to log Events but we don't use the engine API from the hosts for security reasons (to avoid persisting the password on the disk). So this case will be out of the scope of this fix for now.

Code-wise note: I would add an Optional<MigrationReason> on the migration parameters so the API client doesn't have to do extra work after firing the migration. If its there it will be logged prior to contacting VDSM.

Comment 5 Sandro Bonazzola 2016-12-12 14:01:38 UTC
The fix for this issue should be included in oVirt 4.1.0 beta 1 released on December 1st. If not included please move back to modified.

Comment 6 Artyom 2016-12-18 12:09:56 UTC
Verified on ovirt-engine-4.1.0-0.2.master.20161216212250.gitc040969.el7.centos.noarch

Affinity - 2016-12-18 13:57:59,881+02 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler2) [644672e0] Correlation I
D: 644672e0, Job ID: bb6be052-8c5d-44e0-87a8-9a032de2375c, Call Stack: null, Custom Event ID: -1, Message: Migration initiated by system (VM: golden_env_m
ixed_virtio_0, Source: host_mixed_1, Destination: host_mixed_2, Reason: Affinity rules enforcement).

Load balancing - 2016-12-18 14:04:05,201+02 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler7) [4c5000d1] Correlation I
D: 4c5000d1, Job ID: 9d9f216f-3af0-4819-a95a-b8b38c4032bc, Call Stack: null, Custom Event ID: -1, Message: Migration initiated by system (VM: golden_env_m
ixed_virtio_0, Source: host_mixed_2, Destination: host_mixed_1, Reason: Load balancing).