Description of problem: Failed to start VM on PPC twice with error: libvirtError: internal error: process exited while connecting to monitor: 2016-02-03T08:52:53.899885Z qemu-kvm: Failed to allocate HTAB of requested size, try with smaller maxmem Version-Release number of selected component (if applicable): RHEVM Version: 3.6.3-0.1.el6 vdsm-4.17.19-0.el7ev libvirt-1.2.17-13.el7_2.3 How reproducible: happened twice Steps to Reproduce: Start VM Actual results: Failed to start VM Additional info: Engine log: 2016-02-03 10:53:13,236 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-37) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM golden_env_mixed_virtio_1_1 is down with error. Exit message: internal error: process exited while connecting to monitor: 2016-02-03T08:52:53.899885Z qemu-kvm: Failed to allocate HTAB of requested size, try with smaller maxmem vdsm log: Thread-37::DEBUG::2016-02-03 03:52:54,414::lvm::370::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex Thread-37::DEBUG::2016-02-03 03:52:54,414::lvm::290::Storage.Misc.excCmd::(cmd) /usr/bin/taskset --cpu-list 0-127 /usr/bin/sudo -n /usr/sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ '\''a|/dev/mapper/36090a06890b406919966e5b5232bccb6|/dev/mapper/36090a06890b46688996685b5232bdc38|/dev/mapper/36090a06890b4b68d9966b5b5232b0cc0|/dev/mapper/360a98000324669436c2b45666c594b48|/dev/mapper/360a98000324669436c2b45666c594b4a|/dev/mapper/360a98000324669436c2b45666c594b4c|/dev/mapper/360a98000324669436c2b45666c594b4e|/dev/mapper/360a98000324669436c2b45666c594b50|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 0eb900dc-f4fe-4972-b728-ca67bf051dde (cwd None) Thread-27415::ERROR::2016-02-03 03:52:54,557::vm::759::virt.vm::(_startUnderlyingVm) vmId=`a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d`::The vm start process failed Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm self._run() File "/usr/share/vdsm/virt/vm.py", line 1941, in _run self._connection.createXML(domxml, flags), File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 124, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3611, in createXML if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) libvirtError: internal error: process exited while connecting to monitor: 2016-02-03T08:52:53.899885Z qemu-kvm: Failed to allocate HTAB of requested size, try with smaller maxmem Thread-27415::INFO::2016-02-03 03:52:54,559::vm::1330::virt.vm::(setDownStatus) vmId=`a2d23419-e7f6-45d9-b97c-a87bd2c3cd3d`::Changed state to Down: internal error: process exited while connecting to monitor: 2016-02-03T08:52:53.899885Z qemu-kvm: Failed to allocate HTAB of requested size, try with smaller maxmem (code=1)
Created attachment 1120706 [details] engine_log
Created attachment 1120708 [details] vdsm_log
Created attachment 1120709 [details] libvirt_log
update with version: KVM: 2.3.0 - 31.el7_2.7 kernel: 3.10.0 - 327.10.1.el7.ppc64le
Note that memory hotplug is enabled
(In reply to Israel Pinto from comment #5) > Note that memory hotplug is enabled this should behave the same with hotplug disabled if you would use large enough guests The root cause is https://bugzilla.redhat.com/show_bug.cgi?id=1282833#c16 and there is currently no good solution on RHEV side, so this need to be treated as a known issue until it's resolved
(In reply to Michal Skrivanek from comment #6) > (In reply to Israel Pinto from comment #5) > > Note that memory hotplug is enabled > > this should behave the same with hotplug disabled if you would use large > enough guests > The root cause is https://bugzilla.redhat.com/show_bug.cgi?id=1282833#c16 So this bug should depend on bug 1282833 ? And move away from 3.6.4 please. > and there is currently no good solution on RHEV side, so this need to be > treated as a known issue until it's resolved
real fix is tracked in bug 1284775 moving to 4.0 as the bugs 1284775 depends on are heavily in progress and won't be ready before RHEL 7.3
deferred to next version due to dependency on platform bug.
this has been fixed by 4.2 GA by moving to pseries-7.5.0 machine type
Hi Michal. Please review and update the Doc Text as you see fit. In particular, please review the speculative statement, "The current release fixes this issue by accounting for the overhead when scheduling a virtual machine that has a large guest operating system.", which I added based on comment#12.
The doc text is correct about the past state, but the fix is different. it was actually solved by dynamic hash page table resizing (bug 1305398 and bug 1305399 in qemu/kernel).