Bug 862518 - dynamic RHEV guests
Summary: dynamic RHEV guests
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Retired
Component: scheduler
Version: 0.9
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: 0.10.0
Assignee: Dan Callaghan
QA Contact:
URL:
Whiteboard: Cloud
Depends On: 655009
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-10-03 06:25 UTC by Dan Callaghan
Modified: 2018-02-06 00:41 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-11-22 05:57:56 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 880002 0 medium CLOSED Upgrade RHEV from 3.0 to 3.1 2021-02-22 00:41:40 UTC

Internal Links: 880002

Description Dan Callaghan 2012-10-03 06:25:53 UTC
Beaker should integrate with RHEV-M hypervisors to dynamically create a new guest for each recipe (when possible/appropriate).

Comment 1 Min Shin 2012-10-17 05:46:20 UTC
Comments from David Kovalsky on 15/Oct/12: Please sync with Filip Škola on the state of dynamic ESX guests so we have the same interface. Even if the features are developed in parallel, they should be tested / staged together.

Comment 2 Dan Callaghan 2012-10-24 02:35:49 UTC
Most of the work for this feature was already done by Steven Lawrance, but his patch needs some work in light of bug 655009.

Comment 3 Min Shin 2012-10-24 06:10:22 UTC
In 0.10 PRD review request in Vault, bpeck commented:
I would add that RHEVM scheduling should be able to be easily disabled by the admins in case unforeseen issues pop up.  I worry that the new feature could have a negative impact on scheduling if the queue backs up.

Is this use case covered in 0.10?

https://vault-stage.englab.nay.redhat.com/Vault/showRequest/1104

Comment 4 Dan Callaghan 2012-10-24 06:13:46 UTC
(In reply to comment #3)
> In 0.10 PRD review request in Vault, bpeck commented:
> I would add that RHEVM scheduling should be able to be easily disabled by
> the admins in case unforeseen issues pop up.  I worry that the new feature
> could have a negative impact on scheduling if the queue backs up.
> 
> Is this use case covered in 0.10?

That's a good point. The current version of the patch does not have a simple "off" switch. I will need to add something.

Comment 6 Dan Callaghan 2012-10-31 08:06:05 UTC
Have uncovered a few issues with Steve's patch..

Comment 9 Dan Callaghan 2012-11-06 05:11:09 UTC
Setting to ON_DEV as this is now merged, although there are still a few small issues that need to be sorted out.

Comment 10 Dan Callaghan 2012-11-06 05:36:35 UTC
(In reply to comment #9)
> Setting to ON_DEV as this is now merged, although there are still a few
> small issues that need to be sorted out.

Namely:

* typo in destroy_vm
* ks_appends come after install_done therefore never run, due to RHEV reboot
* exception handling in beakerd.virt_recipes explodes if exception happened in flush
* MAC addresses need to be allocated, the same way as for guestrecipes
* RHEL3 may not need to be excluded (RHEV docs claim it is supported, but probably not with virtio devices)

Comment 12 Dan Callaghan 2012-11-07 06:45:39 UTC
(In reply to comment #10)
> * typo in destroy_vm

http://git.beaker-project.org/cgit/beaker/commit/?id=a28a2426cbf

> * ks_appends come after install_done therefore never run, due to RHEV reboot

http://gerrit.beaker-project.org/1470

> * exception handling in beakerd.virt_recipes explodes if exception happened
> in flush

http://gerrit.beaker-project.org/1471

> * MAC addresses need to be allocated, the same way as for guestrecipes

http://gerrit.beaker-project.org/1472

Comment 13 Dan Callaghan 2012-11-08 04:41:05 UTC
A few more issues that need fixing:

* If the kernel or initrd image doesn't exist when we try to start the installation, RHEV silently fails to start the VM. Need to investigate whether the failure is indeed silent or if we can catch it somehow. Not particularly serious since the external watchdog will just get it anyway.

* However... the external watchdog will fail to delete the VM:

2012-11-08 14:25:09,875 bkr.server.model ERROR Failed to destroy vm guest_for_recipe_220, leaked!
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/bkr/server/model.py", line 6005, in release
    manager.destroy_vm(self.system_name)
  File "/usr/lib/python2.6/site-packages/bkr/server/model.py", line 6492, in destroy_vm
    vm.stop()
  File "/usr/lib/python2.6/site-packages/ovirtsdk/infrastructure/brokers.py", line 5313, in stop
    headers={"Correlation-Id":correlation_id})
  File "/usr/lib/python2.6/site-packages/ovirtsdk/infrastructure/proxy.py", line 128, in request
    last=last)
  File "/usr/lib/python2.6/site-packages/ovirtsdk/infrastructure/proxy.py", line 154, in __doRequest
    raise RequestError, response
RequestError: 
status: 400
reason: Bad Request
detail: Cannot stop VM. VM is not running.

Need to handle this case.

Comment 14 Dan Callaghan 2012-11-09 03:47:33 UTC
(In reply to comment #13)
> * However... the external watchdog will fail to delete the VM:

http://gerrit.beaker-project.org/1479

Comment 15 Raymond Mancy 2012-11-22 05:57:56 UTC
Any further issues with dynamic RHEV provisioning can be opened as seperate bugs.

Comment 16 Raymond Mancy 2012-11-22 06:44:03 UTC
This has now been released


Note You need to log in before you can comment on or make changes to this bug.