Bug 1380822

Summary: Some migration options don't apply
Product: [oVirt] vdsm Reporter: Milan Zamazal <mzamazal>
Component: CoreAssignee: Milan Zamazal <mzamazal>
Status: CLOSED CURRENTRELEASE QA Contact: Israel Pinto <ipinto>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.18.15CC: bazulay, bugs, gklein, lsurette, mgoldboi, michal.skrivanek, srevivo, ycui, ykaul
Target Milestone: ovirt-4.0.5Flags: rule-engine: ovirt-4.0.z+
mgoldboi: planning_ack+
michal.skrivanek: devel_ack+
mavital: testing_ack+
Target Release: 4.18.15.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Previously, "compress" and "auto converge" migration settings for new migration policies were ignored and the corresponding migration options were always enabled. This has been fixed and the options should be set as requested by the policies.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-18 07:36:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1362618    

Description Milan Zamazal 2016-09-30 16:42:24 UTC
Description of problem:

When migration compression (or autoconverge) is disabled in Engine, it actually remains enabled for migrations.

Version-Release number of selected component (if applicable):

vdsm 4.8.15
vdsm master

How reproducible:

Always (observable under proper circumstances).

Steps to Reproduce:
1. Select legacy migration policy for a cluster and disable migration compression for that cluster.
2. Start a VM in the cluster.
3. Make the VM busy so that its migration takes long time.
4. Migrate the VM to another host.
5. Look into vdsm.log and check "compressed" items in migration progress reports.

Actual results:

You should see non-zero values of that item from time to time.

Expected results:

"compressed" is always zero (or not present).

Additional info:

Comment 3 Red Hat Bugzilla Rules Engine 2016-09-30 17:35:34 UTC
Bug tickets must have version flags set prior to targeting them to a release. Please ask maintainer to set the correct version flags and only then set the target milestone.

Comment 4 Milan Zamazal 2016-10-11 16:07:08 UTC
There is still a bug in Engine that ignores settings of those options for the legacy migration policy and always sends "false" as their values. That has been reported as a separate bug, see https://bugzilla.redhat.com/1383738.

Due to the Engine bug, this Vdsm fix changes migration behavior for the legacy policy: While previously compression and autoconverge were always enabled, they are always disabled now. That should be a good thing for 4.0 as those options are of dubious value with old QEMU, with compression just consuming CPU time.

Comment 5 Israel Pinto 2016-10-26 11:00:38 UTC
Verify with:
Engine:Red Hat Virtualization Manager Version: 4.0.5.2-0.2.el7ev
Host:
OS Version:RHEL - 7.3 - 6.el7
OS Description:Red Hat Enterprise Linux Server 7.3 Beta (Maipo)
Kernel Version:3.10.0 - 510.el7.x86_64
KVM Version:2.6.0 - 27.el7
LIBVIRT Version:libvirt-2.0.0-10.el7
VDSM Version:vdsm-4.18.13-1.el7ev
SPICE Version:0.12.4 - 19.el7

Steps to Reproduce:
1. Select legacy migration policy for a cluster and disable migration compression for that cluster.
2. Start a VM in the cluster.
3. Migrate the VM to another host.
4. Check vdsm.log that "compressed" is zero.
Results:
compression is disabled, and compres size is zero.
See log: 
#tail -f /var/log/vdsm/vdsm.log | grep compressed
jsonrpc.Executor/3::DEBUG::2016-10-26 13:55:02,518::__init__::522::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'VM.migrate' in bridge with {u'params': {u'tunneled': u'false', u'dstqemu': u'10.35.117.88', u'autoConverge': u'true', u'src': u'virt-nested-vm14.scl.lab.tlv.redhat.com', u'enableGuestEvents': False, u'dst': u'seal08.qa.lab.tlv.redhat.com:54321', u'vmId': u'6adec1d9-519a-4ca7-b3b6-bae6369777a8', u'abortOnError': u'true', u'compressed': u'false', u'method': u'online'}, u'vmID': u'6adec1d9-519a-4ca7-b3b6-bae6369777a8'}
jsonrpc.Executor/3::DEBUG::2016-10-26 13:55:02,519::API::535::vds::(migrate) {u'tunneled': u'false', u'dstqemu': u'10.35.117.88', u'autoConverge': u'true', u'src': u'virt-nested-vm14.scl.lab.tlv.redhat.com', u'enableGuestEvents': False, u'dst': u'seal08.qa.lab.tlv.redhat.com:54321', u'vmId': u'6adec1d9-519a-4ca7-b3b6-bae6369777a8', u'abortOnError': u'true', u'compressed': u'false', u'method': u'online'}
Thread-139592::INFO::2016-10-26 13:55:16,125::migration::706::virt.vm::(monitor_migration) vmId=`6adec1d9-519a-4ca7-b3b6-bae6369777a8`::Migration Progress: 10 seconds elapsed, 83% of data processed, total data: 1096MB, processed data: 499MB, remaining data: 189MB, transfer speed 52MBps, zero pages: 104922MB, compressed: 0MB, dirty rate: -1, memory iteration: -1

Comment 6 Milan Zamazal 2016-10-26 15:00:52 UTC
Please note that `compressed:' is always zero during the first memory iteration (there is nothing to compress yet). There should be *at least* two or three memory iterations to actually check for the presence of compressed bytes.

Comment 7 Israel Pinto 2016-10-27 07:54:37 UTC
output of migration monitor with compression disabled:
jsonrpc.Executor/0::DEBUG::2016-10-27 10:52:01,953::__init__::530::jsonrpc.JsonRpcServer::(_handle_request) Calling 'VM.migrate' in bridge with {'params': {'tunneled': 'false', 'dstqemu': '10.35.161.161', 'autoConverge': 'false', 'src': 'seal08.qa.lab.tlv.redhat.com', 'enableGuestEvents': False, 'dst': 'virt-nested-vm14.scl.lab.tlv.redhat.com:54321', 'vmId': '2f5a56f5-dcee-4553-a602-e090b6b918aa', 'abortOnError': 'true', 'compressed': 'false', 'method': 'online'}, 'vmID': '2f5a56f5-dcee-4553-a602-e090b6b918aa'}
jsonrpc.Executor/0::DEBUG::2016-10-27 10:52:01,954::API::540::vds::(migrate) {'tunneled': 'false', 'dstqemu': '10.35.161.161', 'autoConverge': 'false', 'src': 'seal08.qa.lab.tlv.redhat.com', 'enableGuestEvents': False, 'dst': 'virt-nested-vm14.scl.lab.tlv.redhat.com:54321', 'vmId': '2f5a56f5-dcee-4553-a602-e090b6b918aa', 'abortOnError': 'true', 'compressed': 'false', 'method': 'online'}
Thread-11438::INFO::2016-10-27 10:52:14,651::migration::712::virt.vm::(monitor_migration) vmId=`2f5a56f5-dcee-4553-a602-e090b6b918aa`::Migration Progress: 9 seconds elapsed, 38% of data processed, total data: 6216MB, processed data: 398MB, remaining data: 3867MB, transfer speed 52MBps, zero pages: 500663MB, compressed: 0MB, dirty rate: 0, memory iteration: 1
Thread-11438::INFO::2016-10-27 10:52:24,653::migration::712::virt.vm::(monitor_migration) vmId=`2f5a56f5-dcee-4553-a602-e090b6b918aa`::Migration Progress: 19 seconds elapsed, 47% of data processed, total data: 6216MB, processed data: 918MB, remaining data: 3348MB, transfer speed 52MBps, zero pages: 500663MB, compressed: 0MB, dirty rate: 0, memory iteration: 1
Thread-11438::INFO::2016-10-27 10:52:34,654::migration::712::virt.vm::(monitor_migration) vmId=`2f5a56f5-dcee-4553-a602-e090b6b918aa`::Migration Progress: 29 seconds elapsed, 81% of data processed, total data: 6216MB, processed data: 1345MB, remaining data: 1242MB, transfer speed 52MBps, zero pages: 931589MB, compressed: 0MB, dirty rate: 0, memory iteration: 1
Thread-11438::INFO::2016-10-27 10:52:44,656::migration::712::virt.vm::(monitor_migration) vmId=`2f5a56f5-dcee-4553-a602-e090b6b918aa`::Migration Progress: 40 seconds elapsed, 89% of data processed, total data: 6216MB, processed data: 1860MB, remaining data: 728MB, transfer speed 52MBps, zero pages: 931589MB, compressed: 0MB, dirty rate: 0, memory iteration: 1
Thread-11438::INFO::2016-10-27 10:52:54,657::migration::712::virt.vm::(monitor_migration) vmId=`2f5a56f5-dcee-4553-a602-e090b6b918aa`::Migration Progress: 50 seconds elapsed, 98% of data processed, total data: 6216MB, processed data: 2377MB, remaining data: 177MB, transfer speed 52MBps, zero pages: 940546MB, compressed: 0MB, dirty rate: 0, memory iteration: 1