Bug 1097589 (RHEV_HOT_UNPLUG_CPU)

Summary: [RFE] [7.3] Hot Un-Plug CPU - Support dynamic virtual CPU deallocation
Product: Red Hat Enterprise Virtualization Manager Reporter: meital avital <mavital>
Component: ovirt-engineAssignee: Milan Zamazal <mzamazal>
Status: CLOSED ERRATA QA Contact: Israel Pinto <ipinto>
Severity: medium Docs Contact:
Priority: high    
Version: unspecifiedCC: bcholler, bmcclain, hannsj_uhl, ichute, istein, jcoscia, jraju, juzhang, knoel, lpeer, lsurette, mgoldboi, michal.skrivanek, rbalakri, rgolan, Rhev-m-bugs, srevivo, trichard, yhuang, ykaul
Target Milestone: ovirt-4.1.0-betaKeywords: FutureFeature
Target Release: ---Flags: sherold: Triaged+
bmcclain: priority_rfe_tracking+
ipinto: testing_plan_complete+
Hardware: Unspecified   
OS: Unspecified   
URL: https://trello.com/c/WnlVB3kO
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
This release adds support for CPU hot unplug to Red Hat Virtualization. Note that the guest operating system must also support the feature, and only previously hot plugged CPUs can be hot unplugged.
 Story Points: ---
Clone Of:
: 1097929 1289173 (view as bug list) Environment:
Last Closed: 2017-04-25 00:52:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1087672, 1097929, 1097930, 1146944, 1167336, 1167392    
Bug Blocks: 1289173, 1410090    
Attachments:
Description Flags
engine.log
none
vdsm log none

Description meital avital 2014-05-14 07:00:39 UTC 
Description of problem:
RHEV-M only supports the ability to hot plug vCPUs (Bug 833102).
RHEV-M should allow the admin to dynamically hot remove virtual CPUs from a guest that has this capability.
Comment 3 Roy Golan 2014-05-15 09:36:55 UTC 
self note: make sure the tooltip in UI says Hot add/remove when this is fixed
Comment 7 Michal Skrivanek 2016-02-19 12:56:44 UTC 
needs 7.3 machine type
Comment 8 Michal Skrivanek 2016-09-26 09:10:08 UTC 
should be ready for 4.1
upstream tracking https://trello.com/c/WnlVB3kO
Comment 9 Israel Pinto 2016-12-04 13:55:25 UTC 
Checked with version:
oVirt Engine Version: 4.1.0-0.2.master.20161203231307.gitd7d920b.el7.centos

Steps:
1. Create vm with os 
2. Start VM
3. Hotplug CPU from 1 to 2 CPU
4. Hot un-plug CPU from 2 to 1 CPU
Results:
The hotplug succeed and the hot unplug action is allowed from engine 
but failed.

engine log:
2016-12-04 15:43:38,275+02 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-15) [7706c0e4] Correlation ID: 7706c0e4, Call Stack: null, Custom Event ID: -1, Message: Hotplug CPU: changed the number of CPUs on VM golden_env_mixed_virtio_0 from 1 to 2
2016-12-04 15:43:38,294+02 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-15) [7706c0e4] Correlation ID: 10a3827f-8b87-48e7-a89f-d7f119784dba, Job ID: 764d114d-da88-458f-b285-408b414c025c, Call Stack: null, Custom Event ID: -1, Message: VM golden_env_mixed_virtio_0 configuration was updated by admin@internal-authz.
2016-12-04 15:43:38,299+02 INFO  [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-15) [7706c0e4] Lock freed to object 'EngineLock:{exclusiveLocks='[golden_env_mixed_virtio_0=<VM_NAME, ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]', sharedLocks='[a74698b5-e159-499f-808c-16089f4dbb82=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]'}'
2016-12-04 15:43:38,982+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler4) [554909ae] START, FullListVDSCommand(HostName = host_mixed_3, FullListVDSCommandParameters:{runAsync='true', hostId='498fce9d-c932-43ea-9e13-32f0ec195710', vmIds='[a74698b5-e159-499f-808c-16089f4dbb82]'}), log id: 6852d7b0
2016-12-04 15:43:40,010+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler4) [554909ae] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel7.2.0, vmId=a74698b5-e159-499f-808c-16089f4dbb82, guestDiskMapping={1e05c0b6-7cec-444e-b={name=/dev/vda}, QEMU_DVD-ROM_QM00003={name=/dev/sr0}}, transparentHugePages=true, timeOffset=0, cpuType=Conroe, smp=2, pauseCode=NOERR, guestNumaNodes=[Ljava.lang.Object;@51193ed1, smartcardEnable=false, custom={device_2668f913-1e53-41c6-a0ff-ade90f35bfd9device_cb7680fd-517f-4f01-bd74-b719b788da95=VmDevice:{id='VmDeviceId:{deviceId='cb7680fd-517f-4f01-bd74-b719b788da95', vmId='a74698b5-e159-499f-808c-16089f4dbb82'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_2668f913-1e53-41c6-a0ff-ade90f35bfd9device_cb7680fd-517f-4f01-bd74-b719b788da95device_a0403b0f-7fb0-4545-9b94-e0174c363b97device_6c30d511-42c7-4a31-9f84-3c2c678d0cdb=VmDevice:{id='VmDeviceId:{deviceId='6c30d511-42c7-4a31-9f84-3c2c678d0cdb', vmId='a74698b5-e159-499f-808c-16089f4dbb82'}', device='spicevmc', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=3}', managed='false', plugged='true', readOnly='false', deviceAlias='channel2', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_2668f913-1e53-41c6-a0ff-ade90f35bfd9=VmDevice:{id='VmDeviceId:{deviceId='2668f913-1e53-41c6-a0ff-ade90f35bfd9', vmId='a74698b5-e159-499f-808c-16089f4dbb82'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_2668f913-1e53-41c6-a0ff-ade90f35bfd9device_cb7680fd-517f-4f01-bd74-b719b788da95device_a0403b0f-7fb0-4545-9b94-e0174c363b97=VmDevice:{id='VmDeviceId:{deviceId='a0403b0f-7fb0-4545-9b94-e0174c363b97', vmId='a74698b5-e159-499f-808c-16089f4dbb82'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=golden_env_mixed_virtio_0, nice=0, status=Up, maxMemSize=4194304, bootMenuEnable=false, pid=13012, smpThreadsPerCore=1, memGuaranteedSize=1024, kvmEnable=true, pitReinjection=false, displayNetwork=ovirtmgmt, devices=[Ljava.lang.Object;@70609911, display=qxl, maxVCpus=16, clientIp=, statusTime=4296123560, maxMemSlots=16}], log id: 6852d7b0
2016-12-04 15:43:40,013+02 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring] (DefaultQuartzScheduler4) [554909ae] Received a spice Device without an address when processing VM a74698b5-e159-499f-808c-16089f4dbb82 devices, skipping device: {device=spice, specParams={fileTransferEnable=true, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.162.62, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,ssmartcard,susbredir}, type=graphics, deviceId=cfd00791-3def-4386-938f-547f736e956d, tlsPort=5900}
2016-12-04 15:43:51,094+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10) [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'balloonEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:51,097+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10) [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'rngDevice' can not be updated when status is 'Up'
2016-12-04 15:43:51,099+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10) [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'soundDeviceEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:51,099+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10) [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'consoleEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:51,101+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10) [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'virtioScsiEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:51,102+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10) [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'graphicsDevices' can not be updated when status is 'Up'
2016-12-04 15:43:52,450+02 INFO  [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Lock Acquired to object 'EngineLock:{exclusiveLocks='[golden_env_mixed_virtio_0=<VM_NAME, ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]', sharedLocks='[a74698b5-e159-499f-808c-16089f4dbb82=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]'}'
2016-12-04 15:43:52,507+02 INFO  [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Running command: UpdateVmCommand internal: false. Entities affected :  ID: a74698b5-e159-499f-808c-16089f4dbb82 Type: VMAction group EDIT_VM_PROPERTIES with role type USER
2016-12-04 15:43:52,508+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'balloonEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:52,510+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'rngDevice' can not be updated when status is 'Up'
2016-12-04 15:43:52,511+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'soundDeviceEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:52,521+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'consoleEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:52,522+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'virtioScsiEnabled' can not be updated when status is 'Up'
2016-12-04 15:43:52,524+02 WARN  [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'graphicsDevices' can not be updated when status is 'Up'
2016-12-04 15:43:52,555+02 INFO  [org.ovirt.engine.core.bll.HotSetNumberOfCpusCommand] (default task-1) [2f094958] Running command: HotSetNumberOfCpusCommand internal: true. Entities affected :  ID: a74698b5-e159-499f-808c-16089f4dbb82 Type: VMAction group EDIT_VM_PROPERTIES with role type USER
2016-12-04 15:43:52,558+02 INFO  [org.ovirt.engine.core.vdsbroker.SetNumberOfCpusVDSCommand] (default task-1) [2f094958] START, SetNumberOfCpusVDSCommand(HostName = host_mixed_3, Params:{runAsync='true', hostId='498fce9d-c932-43ea-9e13-32f0ec195710', vmId='a74698b5-e159-499f-808c-16089f4dbb82', numberOfCpus='1'}), log id: 421be49b
2016-12-04 15:43:53,579+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1) [2f094958] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM host_mixed_3 command failed: unsupported configuration: failed to find appropriate hotpluggable vcpus to reach the desired target vcpu count
2016-12-04 15:43:53,579+02 ERROR [org.ovirt.engine.core.vdsbroker.SetNumberOfCpusVDSCommand] (default task-1) [2f094958] Command 'SetNumberOfCpusVDSCommand(HostName = host_mixed_3, Params:{runAsync='true', hostId='498fce9d-c932-43ea-9e13-32f0ec195710', vmId='a74698b5-e159-499f-808c-16089f4dbb82', numberOfCpus='1'})' execution failed: VDSGenericException: VDSErrorException: Failed in vdscommand to SetNumberOfCpusVDS, error = unsupported configuration: failed to find appropriate hotpluggable vcpus to reach the desired target vcpu count
2016-12-04 15:43:53,579+02 INFO  [org.ovirt.engine.core.vdsbroker.SetNumberOfCpusVDSCommand] (default task-1) [2f094958] FINISH, SetNumberOfCpusVDSCommand, log id: 421be49b
2016-12-04 15:43:53,591+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1) [2f094958] Correlation ID: 2f094958, Call Stack: null, Custom Event ID: -1, Message: Failed to hot set number of CPUS to VM golden_env_mixed_virtio_0. Underlying error message: unsupported configuration: failed to find appropriate hotpluggable vcpus to reach the desired target vcpu count


vdsm.log:
2016-12-04 15:43:47,969 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.02 seconds (__init__:515)
2016-12-04 15:43:52,564 ERROR (jsonrpc/6) [virt.vm] (vmId='a74698b5-e159-499f-808c-16089f4dbb82') setNumberOfCpus failed (vm:2528)
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 2526, in setNumberOfCpus
    libvirt.VIR_DOMAIN_AFFECT_CURRENT)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 69, in f
    ret = attr(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 128, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 936, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2589, in setVcpusFlags
    if ret == -1: raise libvirtError ('virDomainSetVcpusFlags() failed', dom=self)
libvirtError: unsupported configuration: failed to find appropriate hotpluggable vcpus to reach the desired target vcpu count
2016-12-04 15:43:52,565 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call VM.setNumberOfCpus failed (error 60) in 0.00 seconds (__init__:515)

attaching  logs
Comment 10 Israel Pinto 2016-12-04 14:00:46 UTC 
Created attachment 1227921 [details]
engine.log
Comment 11 Israel Pinto 2016-12-04 14:01:29 UTC 
Created attachment 1227922 [details]
vdsm log
Comment 12 Israel Pinto 2016-12-05 07:24:05 UTC 
(In reply to Israel Pinto from comment #9)
> Checked with version:
> oVirt Engine Version: 4.1.0-0.2.master.20161203231307.gitd7d920b.el7.centos
> 
> Steps:
> 1. Create vm with os 
> 2. Start VM
> 3. Hotplug CPU from 1 to 2 CPU
> 4. Hot un-plug CPU from 2 to 1 CPU
> Results:
> The hotplug succeed and the hot unplug action is allowed from engine 
> but failed.
> 
> engine log:
> 2016-12-04 15:43:38,275+02 INFO 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (default task-15) [7706c0e4] Correlation ID: 7706c0e4, Call Stack: null,
> Custom Event ID: -1, Message: Hotplug CPU: changed the number of CPUs on VM
> golden_env_mixed_virtio_0 from 1 to 2
> 2016-12-04 15:43:38,294+02 INFO 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (default task-15) [7706c0e4] Correlation ID:
> 10a3827f-8b87-48e7-a89f-d7f119784dba, Job ID:
> 764d114d-da88-458f-b285-408b414c025c, Call Stack: null, Custom Event ID: -1,
> Message: VM golden_env_mixed_virtio_0 configuration was updated by
> admin@internal-authz.
> 2016-12-04 15:43:38,299+02 INFO  [org.ovirt.engine.core.bll.UpdateVmCommand]
> (default task-15) [7706c0e4] Lock freed to object
> 'EngineLock:{exclusiveLocks='[golden_env_mixed_virtio_0=<VM_NAME,
> ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]',
> sharedLocks='[a74698b5-e159-499f-808c-16089f4dbb82=<VM,
> ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]'}'
> 2016-12-04 15:43:38,982+02 INFO 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand]
> (DefaultQuartzScheduler4) [554909ae] START, FullListVDSCommand(HostName =
> host_mixed_3, FullListVDSCommandParameters:{runAsync='true',
> hostId='498fce9d-c932-43ea-9e13-32f0ec195710',
> vmIds='[a74698b5-e159-499f-808c-16089f4dbb82]'}), log id: 6852d7b0
> 2016-12-04 15:43:40,010+02 INFO 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand]
> (DefaultQuartzScheduler4) [554909ae] FINISH, FullListVDSCommand, return:
> [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel7.2.0,
> vmId=a74698b5-e159-499f-808c-16089f4dbb82,
> guestDiskMapping={1e05c0b6-7cec-444e-b={name=/dev/vda},
> QEMU_DVD-ROM_QM00003={name=/dev/sr0}}, transparentHugePages=true,
> timeOffset=0, cpuType=Conroe, smp=2, pauseCode=NOERR,
> guestNumaNodes=[Ljava.lang.Object;@51193ed1, smartcardEnable=false,
> custom={device_2668f913-1e53-41c6-a0ff-ade90f35bfd9device_cb7680fd-517f-4f01-
> bd74-b719b788da95=VmDevice:{id='VmDeviceId:{deviceId='cb7680fd-517f-4f01-
> bd74-b719b788da95', vmId='a74698b5-e159-499f-808c-16089f4dbb82'}',
> device='unix', type='CHANNEL', bootOrder='0', specParams='[]',
> address='{bus=0, controller=0, type=virtio-serial, port=1}',
> managed='false', plugged='true', readOnly='false', deviceAlias='channel0',
> customProperties='[]', snapshotId='null', logicalName='null',
> hostDevice='null'},
> device_2668f913-1e53-41c6-a0ff-ade90f35bfd9device_cb7680fd-517f-4f01-bd74-
> b719b788da95device_a0403b0f-7fb0-4545-9b94-e0174c363b97device_6c30d511-42c7-
> 4a31-9f84-3c2c678d0cdb=VmDevice:{id='VmDeviceId:{deviceId='6c30d511-42c7-
> 4a31-9f84-3c2c678d0cdb', vmId='a74698b5-e159-499f-808c-16089f4dbb82'}',
> device='spicevmc', type='CHANNEL', bootOrder='0', specParams='[]',
> address='{bus=0, controller=0, type=virtio-serial, port=3}',
> managed='false', plugged='true', readOnly='false', deviceAlias='channel2',
> customProperties='[]', snapshotId='null', logicalName='null',
> hostDevice='null'},
> device_2668f913-1e53-41c6-a0ff-ade90f35bfd9=VmDevice:{id='VmDeviceId:
> {deviceId='2668f913-1e53-41c6-a0ff-ade90f35bfd9',
> vmId='a74698b5-e159-499f-808c-16089f4dbb82'}', device='ide',
> type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01,
> bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false',
> plugged='true', readOnly='false', deviceAlias='ide', customProperties='[]',
> snapshotId='null', logicalName='null', hostDevice='null'},
> device_2668f913-1e53-41c6-a0ff-ade90f35bfd9device_cb7680fd-517f-4f01-bd74-
> b719b788da95device_a0403b0f-7fb0-4545-9b94-e0174c363b97=VmDevice:
> {id='VmDeviceId:{deviceId='a0403b0f-7fb0-4545-9b94-e0174c363b97',
> vmId='a74698b5-e159-499f-808c-16089f4dbb82'}', device='unix',
> type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0,
> controller=0, type=virtio-serial, port=2}', managed='false', plugged='true',
> readOnly='false', deviceAlias='channel1', customProperties='[]',
> snapshotId='null', logicalName='null', hostDevice='null'}}, vmType=kvm,
> memSize=1024, smpCoresPerSocket=1, vmName=golden_env_mixed_virtio_0, nice=0,
> status=Up, maxMemSize=4194304, bootMenuEnable=false, pid=13012,
> smpThreadsPerCore=1, memGuaranteedSize=1024, kvmEnable=true,
> pitReinjection=false, displayNetwork=ovirtmgmt,
> devices=[Ljava.lang.Object;@70609911, display=qxl, maxVCpus=16, clientIp=,
> statusTime=4296123560, maxMemSlots=16}], log id: 6852d7b0
> 2016-12-04 15:43:40,013+02 INFO 
> [org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring]
> (DefaultQuartzScheduler4) [554909ae] Received a spice Device without an
> address when processing VM a74698b5-e159-499f-808c-16089f4dbb82 devices,
> skipping device: {device=spice, specParams={fileTransferEnable=true,
> displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.162.62,
> spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,
> ssmartcard,susbredir}, type=graphics,
> deviceId=cfd00791-3def-4386-938f-547f736e956d, tlsPort=5900}
> 2016-12-04 15:43:51,094+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10)
> [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'balloonEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:51,097+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10)
> [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'rngDevice' can not be updated
> when status is 'Up'
> 2016-12-04 15:43:51,099+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10)
> [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'soundDeviceEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:51,099+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10)
> [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'consoleEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:51,101+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10)
> [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'virtioScsiEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:51,102+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10)
> [e719f5a6-0f09-490b-8aad-9b547ddf1f57] Field 'graphicsDevices' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:52,450+02 INFO  [org.ovirt.engine.core.bll.UpdateVmCommand]
> (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Lock Acquired to
> object 'EngineLock:{exclusiveLocks='[golden_env_mixed_virtio_0=<VM_NAME,
> ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]',
> sharedLocks='[a74698b5-e159-499f-808c-16089f4dbb82=<VM,
> ACTION_TYPE_FAILED_VM_IS_BEING_UPDATED>]'}'
> 2016-12-04 15:43:52,507+02 INFO  [org.ovirt.engine.core.bll.UpdateVmCommand]
> (default task-1) [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Running command:
> UpdateVmCommand internal: false. Entities affected :  ID:
> a74698b5-e159-499f-808c-16089f4dbb82 Type: VMAction group EDIT_VM_PROPERTIES
> with role type USER
> 2016-12-04 15:43:52,508+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1)
> [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'balloonEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:52,510+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1)
> [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'rngDevice' can not be updated
> when status is 'Up'
> 2016-12-04 15:43:52,511+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1)
> [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'soundDeviceEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:52,521+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1)
> [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'consoleEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:52,522+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1)
> [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'virtioScsiEnabled' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:52,524+02 WARN 
> [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-1)
> [7a759769-f218-4ad3-b6d1-33f7a8ccab26] Field 'graphicsDevices' can not be
> updated when status is 'Up'
> 2016-12-04 15:43:52,555+02 INFO 
> [org.ovirt.engine.core.bll.HotSetNumberOfCpusCommand] (default task-1)
> [2f094958] Running command: HotSetNumberOfCpusCommand internal: true.
> Entities affected :  ID: a74698b5-e159-499f-808c-16089f4dbb82 Type: VMAction
> group EDIT_VM_PROPERTIES with role type USER
> 2016-12-04 15:43:52,558+02 INFO 
> [org.ovirt.engine.core.vdsbroker.SetNumberOfCpusVDSCommand] (default task-1)
> [2f094958] START, SetNumberOfCpusVDSCommand(HostName = host_mixed_3,
> Params:{runAsync='true', hostId='498fce9d-c932-43ea-9e13-32f0ec195710',
> vmId='a74698b5-e159-499f-808c-16089f4dbb82', numberOfCpus='1'}), log id:
> 421be49b
> 2016-12-04 15:43:53,579+02 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (default task-1) [2f094958] Correlation ID: null, Call Stack: null, Custom
> Event ID: -1, Message: VDSM host_mixed_3 command failed: unsupported
> configuration: failed to find appropriate hotpluggable vcpus to reach the
> desired target vcpu count
> 2016-12-04 15:43:53,579+02 ERROR
> [org.ovirt.engine.core.vdsbroker.SetNumberOfCpusVDSCommand] (default task-1)
> [2f094958] Command 'SetNumberOfCpusVDSCommand(HostName = host_mixed_3,
> Params:{runAsync='true', hostId='498fce9d-c932-43ea-9e13-32f0ec195710',
> vmId='a74698b5-e159-499f-808c-16089f4dbb82', numberOfCpus='1'})' execution
> failed: VDSGenericException: VDSErrorException: Failed in vdscommand to
> SetNumberOfCpusVDS, error = unsupported configuration: failed to find
> appropriate hotpluggable vcpus to reach the desired target vcpu count
> 2016-12-04 15:43:53,579+02 INFO 
> [org.ovirt.engine.core.vdsbroker.SetNumberOfCpusVDSCommand] (default task-1)
> [2f094958] FINISH, SetNumberOfCpusVDSCommand, log id: 421be49b
> 2016-12-04 15:43:53,591+02 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (default task-1) [2f094958] Correlation ID: 2f094958, Call Stack: null,
> Custom Event ID: -1, Message: Failed to hot set number of CPUS to VM
> golden_env_mixed_virtio_0. Underlying error message: unsupported
> configuration: failed to find appropriate hotpluggable vcpus to reach the
> desired target vcpu count
> 
> 
> vdsm.log:
> 2016-12-04 15:43:47,969 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call
> StoragePool.getInfo succeeded in 0.02 seconds (__init__:515)
> 2016-12-04 15:43:52,564 ERROR (jsonrpc/6) [virt.vm]
> (vmId='a74698b5-e159-499f-808c-16089f4dbb82') setNumberOfCpus failed
> (vm:2528)
> Traceback (most recent call last):
>   File "/usr/share/vdsm/virt/vm.py", line 2526, in setNumberOfCpus
>     libvirt.VIR_DOMAIN_AFFECT_CURRENT)
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 69,
> in f
>     ret = attr(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line
> 128, in wrapper
>     ret = f(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 936, in wrapper
>     return func(inst, *args, **kwargs)
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2589, in
> setVcpusFlags
>     if ret == -1: raise libvirtError ('virDomainSetVcpusFlags() failed',
> dom=self)
> libvirtError: unsupported configuration: failed to find appropriate
> hotpluggable vcpus to reach the desired target vcpu count
> 2016-12-04 15:43:52,565 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call
> VM.setNumberOfCpus failed (error 60) in 0.00 seconds (__init__:515)
> 
> attaching  logs

I also checked the following scenario:
1. Start VM with 4 CPU
2. Unplug  2 CPU 

Same result unplug failed.
Adding host version:
OS Version:RHEL - 7.3 - 7.el7
Kernel Version:3.10.0 - 514.el7.x86_64
KVM Version:2.6.0 - 27.el7
LIBVIRT Version:libvirt-2.0.0-10.el7
VDSM Version:vdsm-4.18.999-1063.gita8cfafc.el7.centos
Comment 13 Milan Zamazal 2016-12-05 15:34:01 UTC 
Please note 7.3 machine type is required. We still use 7.2 by default, see https://gerrit.ovirt.org/65979.

> I also checked the following scenario:
> 1. Start VM with 4 CPU
> 2. Unplug  2 CPU

This can't work: Only hotplugged CPUs can be hotunplugged.
Comment 14 Israel Pinto 2016-12-06 07:20:31 UTC 
Recheck the scenario of hot plug and unplug with machine type: 
pc-i440fx-rhel7.3.0 and it pass.
Comment 15 Michal Skrivanek 2016-12-06 07:49:13 UTC 
7.3 machine type added to 4.1 Cluster Level by https://gerrit.ovirt.org/#/c/65979/,now merged
Comment 16 Israel Pinto 2017-01-22 13:03:53 UTC 
Verify with:
Engine:
oVirt Engine Version: 4.1.0.3-0.0.master.20170122091652.gitc6fc2c2.el7.centos
Host:
OS Version:RHEL - 7.3 - 7.el7
Kernel Version:3.10.0 - 514.el7.x86_64
KVM Version:2.6.0 - 28.el7_3.3.1
LIBVIRT Version:libvirt-2.0.0-10.el7_3.4
VDSM Version:vdsm-4.19.2-2.el7ev
SPICE Version:0.12.4 - 20.el7_3

Test results:
https://polarion.engineering.redhat.com/polarion/#/project/RHEVM3/testrun?id=41%5F1&tab=records&result=passed