Bug 1269534 - [REST-API] cloud-init image does not include cloud-init user defined input when vm is started via API
[REST-API] cloud-init image does not include cloud-init user defined input wh...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.5.5
Unspecified Unspecified
medium Severity medium
: ovirt-3.5.6
: 3.5.6
Assigned To: Shahar Havivi
Israel Pinto
virt
: Automation, AutomationBlocker, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-07 10:04 EDT by sefi litmanovich
Modified: 2016-04-21 01:48 EDT (History)
19 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-01 14:06:23 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
engine and vdsm logs (997.03 KB, application/x-gzip)
2015-10-07 10:04 EDT, sefi litmanovich
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 47325 ovirt-engine-3.5 MERGED RunVmCommand not loading defalut VmInit Never

  None (edit)
Description sefi litmanovich 2015-10-07 10:04:52 EDT
Created attachment 1080691 [details]
engine and vdsm logs

Description of problem:

There seems to be some difference in behaviour between Start vm on REST-API and start vm via webadmin.
The symptom I get relates to cloud-init.
The flow was:
1. upload rhel-guest-image to rhevm and import it as template.
2. create vm from the template.
3. edit vm and set cloud-init initialization with some username and password.
4. start vm (not runOnce).

Actual results:

1. if start vm is invoked via web admin, the VM starts as expected, cloud-init runs it's configurations and finally vm is accessible with chosen username and password.

2. if start vm is invoked via REST-API, the VM starts but cloud-init fails as it appears there's no device (payload) found to use for cloud-init. after sometime vm start up is done but username and password aren't set and vm isn't accessible. It seems that for some reason the cloud-init configuration is override.  

Expected results:

Both scenarios should work the same - VM should start and cloud-init should work and set all it's configurations as expected.


Version-Release number of selected component (if applicable):

rhevm-3.5.5-0.1.el6ev.noarch

How reproducible:
always


Additional info:
Comment 1 Juan Hernández 2015-10-07 10:31:22 EDT
In general the RESTAPI doesn't need to behave like the the GUI.

In this particular case the decision to use or not cloud-init (or sysprep) is made by the user, not by the system. In the GUI you have a check box, in the RESTAPI you need to explicitly use the "use_cloud_init" or "use_sysprep" elements:

  POST /vms/{vm:id}/start
  <action>
    <use_cloud_init>true|false</use_cloud_init>
  </action>

In the absence of this parameter cloud-init (or sysprep) isn't triggered at all.
Comment 2 sefi litmanovich 2015-10-07 10:34:56 EDT
Is this a new behaviour? because in my test I do not pass this parameter and automation was successful until recently (September).
If it is a new tag then I would say close this bug, if not then it's quite strange I hadn't experienced this sooner.
Comment 3 Juan Hernández 2015-10-07 10:35:48 EDT
The "user_cloud_init" element was introduced in order to fix bug 1190663.
Comment 4 sefi litmanovich 2015-10-08 04:35:52 EDT
Tried to add the use_cloud_init tag to the action body message:

<action>
    <async>false</async>
    <grace_period>
        <expiry>10</expiry>
    </grace_period>
    <use_cloud_init>true</use_cloud_init>
</action>

response is 200 so no syntax problem.
VM still fails to initialize with cloud-init.
Is the fix in bz - 1190663 for 3.5 as well?
Let me know if this should be re-named as cloud-init bug instead of REST bug.
Comment 5 Juan Hernández 2015-10-08 12:39:35 EDT
Yes, the fix should be included included in 3.5.5, see bug 1240900. I'm trying to reproduce it, don't change anything yet.
Comment 6 Juan Hernández 2015-10-08 13:55:10 EDT
I have tested this again with 3.5.5 and it works correctly, at least it triggers the creation of the cloud-init ISO image, which is then attached to the VM. Can you verify that? Start the VM using the "use_cloud_init" element, as you did, and then go to the hypervisor and check if the "qemu-kvm" process is running:

  # ps -ef | grep qemu-kvm | grep the_name_of_the_vm

That process, if it exists, should have an option like this:

  --drive file=/var/run/vdsm/payload/....img

That .img file is the ISO image generated by VDSM. You can mount and inspect its content:

   # cp /var/run/vdsm/payload/....img /tmp/cloud-init.img
   # mount -o loop,ro cloud-init.img /mnt
   # find /mnt
   # cat /mnt/openstack/latest/user_data

If the image is generated correctly, and the content is also correct, then there may be an issue with cloud-init itself.
Comment 7 sefi litmanovich 2015-10-12 04:08:42 EDT
The issue is with creation of cloud-init image via API on 3.5.5.
When created via API the image doesn't include the input of cloud init parameters (in my case username and password), whereas using webadmin the image does include the extra parameters.
This doesn't re produce on 3.6.
Comment 8 Shahar Havivi 2015-10-14 08:37:20 EDT
BTW,
As a workaround one can use run-once to run the cloud-init by adding <vm/> tag to the action, ie:
<action>
    <use_cloud_init>true</use_cloud_init>
    <vm/>
</action>
Comment 9 Eyal Edri 2015-10-28 09:37:43 EDT
adding z-stream keyworkd to make bot happy
Comment 13 Israel Pinto 2015-11-05 04:47:10 EST
Verify with:
Engine:rhevm-3.5.6.1-0.1.el6ev
Hosts:
OS: RHEL - 7.2 - 9.el7
Kernal: 3.10.0 - 327.el7.x86_64
KVM: 2.3.0 - 31.el7
VDSM:vdsm-4.16.28-1.el7ev
libvirt: libvirt-1.2.17-13.el7
rhel-guest-image:rhel-guest-image-7.2-20151102.0.x86_64

Scenario:
1. Upload rhel-guest-image to rhevm and import it as template.
2. Create vm from the template.
3. Edit vm and set cloud-init initialization with some username and password.
4. Start vm: Via GUI and REST API
5. Connect to VM via spice: check the user/pass as configure in clould-init
REST request:
 POST /vms/{vm:id}/start
  <action>
    <use_cloud_init>true|false</use_cloud_init>
  </action>

Results:
GUI: Pass
REST API: PASS
Comment 14 Brandon Dunne 2015-11-20 14:35:37 EST
Do I understand correctly that even when passing <use_cloud_init>true</use_cloud_init> to the start command, this will not work on RHEVM 3.5.5?  When I try to test this on RHEVM 3.5.5, the generated payload does not contain any of the initialization data that I set.
Comment 16 Brandon Dunne 2015-11-23 11:28:49 EST
Can you help with: https://bugzilla.redhat.com/show_bug.cgi?id=1269534#c14
Comment 17 Michal Skrivanek 2015-11-24 03:12:00 EST
(In reply to Brandon Dunne from comment #16)
> Can you help with: https://bugzilla.redhat.com/show_bug.cgi?id=1269534#c14

sorry, only 3.5.6 has the complete fix
Comment 22 Brandon Dunne 2015-11-30 10:13:25 EST
Shahar,

My process is:
1. Create VM via REST API
2. Update Initialization via REST API
3. Start VM via REST API (for the first time) with <use_cloud_init>true</use_cloud_init>

Result:
Initialization information is not included in the payload attached to the VM.

This leads me to believe that it is not that same issue as mentioned in Comment 20 above, since that sounds like the VM has been booted once, then initialization information set, then booted a second time.  My VM is only booted once.
Comment 23 Shahar Havivi 2015-12-01 02:11:53 EST
(In reply to Brandon Dunne from comment #22)
The bug that Israel open was regarding running vm with run once didn't include the username and password in cloud-init.
We close the bug because its description was not clear.
Israel will open a new bug regarding that and I will send it to you.

This bug is related that the <use_cloud_init> tag was not working at all.
Comment 24 Brandon Dunne 2015-12-01 07:50:52 EST
Shahar,

I see that starting the VM with <use_cloud_init> is working since a payload is generated and attached, however the payload does not contain any of the initialization data.  Also, I am not using "run once" mode when starting the machine.
Comment 25 Shahar Havivi 2015-12-01 09:15:55 EST
(In reply to Brandon Dunne from comment #24)
I know this is the issue that I talked about.
We are opening a bug regarding that,
Israel will post you the bug #
Comment 26 Shahar Havivi 2015-12-01 09:49:54 EST
I know this is the issue that I talked about.
We are opening a bug regarding that,
Israel will post you the bug #
Comment 27 Shahar Havivi 2015-12-01 09:51:14 EST
Brandon,
is bz 1287122 the same as you reported?
Comment 28 Brandon Dunne 2015-12-01 12:34:01 EST
Shahar,

No, 1287122 is not the same issue.  I set the initialization data on the vm, then boot the VM (not Run Once) with <use_cloud_init> and none of the initialization data appears in the payload.
Comment 29 errata-xmlrpc 2015-12-01 14:06:23 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2531.html
Comment 30 Shahar Havivi 2015-12-02 02:22:20 EST
(In reply to Brandon Dunne from comment #28)
> Shahar,
> 
> No, 1287122 is not the same issue.  I set the initialization data on the vm,
> then boot the VM (not Run Once) with <use_cloud_init> and none of the
> initialization data appears in the payload.
if its not the first run of the VM you have to use run-once
Comment 31 Brandon Dunne 2015-12-02 07:20:22 EST
I am starting the VM for the first time, but not using "Run Once" mode, just "Run" with <use_cloud_init>.  Is there something more that I can do to debug why the payload doesn't contain the initialization data?
Comment 32 Shahar Havivi 2015-12-02 07:39:37 EST
I think that you put the use_cloud_init inside the vm and not inside the action.

this should work:

<action>
    <use_cloud_init>true</use_cloud_init>
  <vm>
    <initialization>

    </initialization>
  </vm>
</action>
Comment 33 Brandon Dunne 2015-12-02 07:52:10 EST
I originally tested with <use_cloud_init> inside <vm> and saw that the payload does not get generated at all.

So, I put the <use_cloud_init> in the <action> and see that a payload is generated but does not contain the initialization data that I already have set on the VM.
Comment 34 Brandon Dunne 2015-12-04 18:01:28 EST
Israel,

Are you able to reproduce this problem?
Comment 36 Brandon Dunne 2015-12-11 12:58:34 EST
Retesting with the released RHEVM 3.5.6 packages was successful.  Clearing my NEEDINFOs

Note You need to log in before you can comment on or make changes to this bug.