Bug 1238390

Summary: cloud-init parameters not being passed to rhev
Product: Red Hat CloudForms Management Engine Reporter: Josh Carter <jocarter>
Component: AutomateAssignee: Brandon Dunne <bdunne>
Status: CLOSED ERRATA QA Contact: Pete Savage <psavage>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.3.0CC: bascar, bazulay, bdunne, cpelland, dajohnso, dornelas, fdewaley, jhardy, jocarter, mfeifer, michal.skrivanek, mkanoor, obarenbo, psavage, simaishi, tfitzger
Target Milestone: GAKeywords: ZStream
Target Release: 5.5.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: 5.5.0.1 Doc Type: Bug Fix
Doc Text:
Cause: Different versions of RHEV handled cloud-init differently. Consequence: Cloud-init parameters not being passed to rhev Fix: Add separate #configure_cloud_init method for Provisioning Ovirt / RHEV. Result: Cloud-init parameters are passed.
Story Points: ---
Clone Of:
: 1249151 (view as bug list) Environment:
Last Closed: 2015-12-08 13:20:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1249151    

Description Josh Carter 2015-07-01 18:32:30 UTC
Description of problem:

Rhev provision using cloud-init does not pass the data from the customize tab to cloud-init. 
The VM deploys normally without issues. 

Root password
Hostname
IP
MASK
GW
DNS
DNS search



Version-Release number of selected component (if applicable): 5.3.4


How reproducible:
very

Steps to Reproduce:

1. make rhev template
2. Delete the keys rm -rf /etc/ssh/ssh_host_*
3. Delete content of /etc/udev/rules.d/70-persistent-net.rules
4. Delete Hardware Address (MAC)from the configuration of /etc/sysconfig/network-scripts/ifcfg-eth0
5. Install the Cloud init package: yum install cloud-init ( from this channel rhel-6-server-rh-common-rpms Red Hat Enterprise Linux 6 Server - RH Common (RPMs) )
6. power off
7. Via RHEV GUI make template
8. Modify Template: Option Initial Execution >> Use Cloud-init/sysprep >> Enable Check Buttom
9. deploy vm based off of the template and provide {Root password,Hostname,IP, MASK, GW, DNS, DNS search} on the customize tab

Actual results:
anything provided within the customize tab is not passed to the vm

Expected results:
 customize tab options should be passed to cloud-init and applied to the vm.


Additional info:
I also have reproduce this issue on 5.4.0.5

Comment 2 Greg McCullough 2015-07-02 16:08:46 UTC
Are you resolving these variables into your cloud_init script?  Can you provide the cloud_init script that you are using with the provision request?

Comment 3 Dave Johnson 2015-07-02 19:11:49 UTC
Pete, can you take a peek at this one, I believe you added an automated test for this already, right?

Comment 5 Greg McCullough 2015-07-09 16:16:20 UTC
Josh - Please specify what version of rhev was being used.  I have a feeling that might be playing a role here.

Comment 6 Greg McCullough 2015-07-10 21:36:39 UTC
In advanced settings set the "level_rhevm: info" value to debug.  Running with this setting should show the data being passed to rhevm.

Comment 7 Brandon Dunne 2015-07-24 21:38:10 UTC
Hi Derrick,

I have a few RHEV questions and was wondering if you could answer them and verify assumptions.

It appears that VM Initialization started working on RHEV 3.4 even though it isn't documented until 3.5 [1].  We used to use the floppy payload method of attaching cloud-init user data, but I haven't had much success in getting that working.  Initialization is the preferred way of sending cloud-init user data, correct?

In the past I remember cloud-init supporting both a cloud-config script and a bash script, however it seems that it now only supports only cloud-config.  Is that correct?

[1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.5/html/Technical_Guide/chap-Virtual_Machines.html

Comment 8 Derrick Ornelas 2015-07-27 20:56:31 UTC
(In reply to Brandon Dunne from comment #7)
> Hi Derrick,
> 
> I have a few RHEV questions and was wondering if you could answer them and
> verify assumptions.
> 
> It appears that VM Initialization started working on RHEV 3.4 even though it
> isn't documented until 3.5 [1].  We used to use the floppy payload method of
> attaching cloud-init user data, but I haven't had much success in getting
> that working.  Initialization is the preferred way of sending cloud-init
> user data, correct?
> 
> In the past I remember cloud-init supporting both a cloud-config script and
> a bash script, however it seems that it now only supports only cloud-config.
> Is that correct?
> 
> [1]
> https://access.redhat.com/documentation/en-US/
> Red_Hat_Enterprise_Virtualization/3.5/html/Technical_Guide/chap-
> Virtual_Machines.html

Unfortunately, I don't know enough about cloud-init to say for sure.  After talking to Josh it sounds like your main issue might be about how RHEV provides cloud-init data to a VM.  For that I suggest talking to Juan (jhernand) as he is the developer for the RHEV REST API.

Comment 9 CFME Bot 2015-07-27 21:23:45 UTC
New commit detected on ovirt/master:
https://github.com/ManageIQ/ovirt/commit/9d40df158f2ba362d046a9788bbe259012a1a425

commit 9d40df158f2ba362d046a9788bbe259012a1a425
Author:     Brandon Dunne <bdunne>
AuthorDate: Thu Jul 23 12:45:04 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Fri Jul 24 16:28:43 2015 -0400

    Add Usage Error exceptions
    
    Raise UsageError when there is a usage_message and no fault
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 lib/ovirt/exception.rb |  3 +++
 lib/ovirt/service.rb   |  6 ++++--
 spec/service_spec.rb   | 18 ++++++++++++++++++
 3 files changed, 25 insertions(+), 2 deletions(-)

Comment 10 CFME Bot 2015-07-27 21:23:50 UTC
New commit detected on ovirt/master:
https://github.com/ManageIQ/ovirt/commit/685531ed38fc2a439aed38e1a4bd4c403a3b80fb

commit 685531ed38fc2a439aed38e1a4bd4c403a3b80fb
Author:     Brandon Dunne <bdunne>
AuthorDate: Fri Jul 24 16:24:20 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Mon Jul 27 14:10:40 2015 -0400

    Add a Vm factory as a base for vm_full
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 spec/factories/vm.rb | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Comment 11 CFME Bot 2015-07-27 21:23:55 UTC
New commit detected on ovirt/master:
https://github.com/ManageIQ/ovirt/commit/bdc54f719f62cffbce34b3621897e084202f9079

commit bdc54f719f62cffbce34b3621897e084202f9079
Author:     Brandon Dunne <bdunne>
AuthorDate: Fri Jul 24 16:27:49 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Mon Jul 27 14:10:46 2015 -0400

    Add Vm#cloud_init= method with support for older versions
    
    Old versions would attach a floppy with the content, new versions (3.4+)
    use vm initialization attributes
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 .../cloud_init_via_floppy_payload.rb               |  7 ++++
 lib/ovirt/vm.rb                                    | 22 +++++++++++++
 .../cloud_init_via_floppy_payload_spec.rb          | 14 ++++++++
 spec/vm_spec.rb                                    | 37 ++++++++++++++++++++++
 4 files changed, 80 insertions(+)
 create mode 100644 lib/ovirt/legacy_support/cloud_init_via_floppy_payload.rb
 create mode 100644 spec/legacy_support/cloud_init_via_floppy_payload_spec.rb

Comment 12 CFME Bot 2015-07-27 21:24:00 UTC
New commit detected on ovirt/master:
https://github.com/ManageIQ/ovirt/commit/8e5fbacecef10620ae7339d0a7ab1464e864d600

commit 8e5fbacecef10620ae7339d0a7ab1464e864d600
Author:     Brandon Dunne <bdunne>
AuthorDate: Mon Jul 27 12:34:24 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Mon Jul 27 14:10:46 2015 -0400

    Convert cloud-config scripts to format expected by Ovirt
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 lib/ovirt/vm.rb | 18 +++++++++++++++---
 spec/vm_spec.rb | 13 ++++++++++---
 2 files changed, 25 insertions(+), 6 deletions(-)

Comment 13 Brandon Dunne 2015-07-28 14:25:14 UTC
Hi Barak,

Are you able to answer the questions in Comment 7?

I was referred to jhernand, but that e-mail isn't registered in bugzilla and when I e-mailed him privately I got a response that he is away.

Thanks,
Brandon

Comment 15 Michal Skrivanek 2015-07-28 15:53:34 UTC
(In reply to Brandon Dunne from comment #7)
> Hi Derrick,
> 
> I have a few RHEV questions and was wondering if you could answer them and
> verify assumptions.
> 
> It appears that VM Initialization started working on RHEV 3.4 even though it
> isn't documented until 3.5 [1].  We used to use the floppy payload method of
> attaching cloud-init user data, but I haven't had much success in getting
> that working.  Initialization is the preferred way of sending cloud-init
> user data, correct?

Yes. Cloud-init is supported since 3.3, but with various issues. 3.5 has the right support for custom scripts. Indeed that would be the preferred way. 
If you are interested in 3.4 then there were few issues during .z so depends which version.

> 
> In the past I remember cloud-init supporting both a cloud-config script and
> a bash script, however it seems that it now only supports only cloud-config.
> Is that correct?

No bash script, additional section in cloudinit data. From there you can run shell stuff via cloudinit syntax

If you would need/want additional files to be pushed together, we can work on that. Let me know
> 
> [1]
> https://access.redhat.com/documentation/en-US/
> Red_Hat_Enterprise_Virtualization/3.5/html/Technical_Guide/chap-
> Virtual_Machines.html

Comment 16 CFME Bot 2015-07-29 18:15:07 UTC
New commit detected on manageiq/master:
https://github.com/ManageIQ/manageiq/commit/5bfe56bb3d467439cbee185825e6c83ca3227514

commit 5bfe56bb3d467439cbee185825e6c83ca3227514
Author:     Brandon Dunne <bdunne>
AuthorDate: Thu Jul 23 09:50:01 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Fri Jul 24 20:22:25 2015 -0400

    Add separate #configure_cloud_init method for Provisioning Ovirt / RHEV
    
    This will allow the ovirt gem to choose the method of attaching the cloud
    init script to the VM based on the Ovirt version and abstract those
    details from ManageIQ
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 .../redhat/infra_manager/provision/configuration.rb  | 20 ++++++++++++--------
 .../redhat/infra_manager/provision/state_machine.rb  |  2 +-
 gems/pending/Gemfile                                 |  2 +-
 .../infra_provider/provision/configuration_spec.rb   | 17 +++++++++++++++--
 .../infra_provider/provision/state_machine_spec.rb   |  2 +-
 5 files changed, 30 insertions(+), 13 deletions(-)

Comment 17 CFME Bot 2015-07-29 18:15:11 UTC
New commit detected on manageiq/master:
https://github.com/ManageIQ/manageiq/commit/65d159070eaa2f7c8071fb0171efcd7bb3b69f3e

commit 65d159070eaa2f7c8071fb0171efcd7bb3b69f3e
Author:     Brandon Dunne <bdunne>
AuthorDate: Fri Jul 24 20:19:07 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Fri Jul 24 20:22:25 2015 -0400

    Remove extra methods in configure/set cpu, memory, etc in RHEV provision
    
    Refactor and update configuration tests
    Test more of #configure_container
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 .../infra_manager/provision/configuration.rb       | 27 +-------
 .../provision/configuration/container.rb           | 38 +++++++----
 spec/factories/host.rb                             |  1 +
 .../infra_provider/provision/configuration_spec.rb | 74 ++++++++++++----------
 4 files changed, 66 insertions(+), 74 deletions(-)

Comment 18 CFME Bot 2015-07-29 22:26:57 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=3174b5fcbd78c08dc1e2fb74d123a8c15c99445a

commit 3174b5fcbd78c08dc1e2fb74d123a8c15c99445a
Author:     Brandon Dunne <bdunne>
AuthorDate: Thu Jul 23 09:50:01 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Wed Jul 29 16:18:19 2015 -0400

    Add separate #configure_cloud_init method for Provisioning Ovirt / RHEV
    
    This will allow the ovirt gem to choose the method of attaching the cloud
    init script to the VM based on the Ovirt version and abstract those
    details from ManageIQ
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 lib/Gemfile                                          |  2 +-
 .../app/models/miq_provision_redhat/configuration.rb | 20 ++++++++++++--------
 .../app/models/miq_provision_redhat/state_machine.rb |  2 +-
 .../miq_provision_redhat/configuration_spec.rb       | 19 ++++++++++++++++---
 .../miq_provision_redhat/state_machine_spec.rb       |  2 +-
 5 files changed, 31 insertions(+), 14 deletions(-)

Comment 19 CFME Bot 2015-07-29 22:27:01 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=df7f01abf15f80da0d6bf176ec0a8080bdc127ec

commit df7f01abf15f80da0d6bf176ec0a8080bdc127ec
Author:     Brandon Dunne <bdunne>
AuthorDate: Wed Jul 29 17:11:58 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Wed Jul 29 17:11:58 2015 -0400

    Remove extra methods in configure/set cpu, memory, etc in RHEV provision
    
    Refactor and update configuration tests
    Test more of #configure_container
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1238390

 .../models/miq_provision_redhat/configuration.rb   | 27 +-------
 .../configuration/container.rb                     | 44 ++++++++-----
 vmdb/spec/factories/host.rb                        |  1 +
 .../miq_provision_redhat/configuration_spec.rb     | 73 +++++++++++-----------
 4 files changed, 68 insertions(+), 77 deletions(-)

Comment 21 Brandon Dunne 2015-09-30 14:00:04 UTC
It would be a significant amount of work to backport to 5.3.z and probably is not worth the risk.

Comment 23 Brandon Dunne 2015-11-25 18:02:13 UTC
Pete, keep this in mind when testing on RHEV 3.5.5+: https://bugzilla.redhat.com/show_bug.cgi?id=1282618

Comment 24 Pete Savage 2015-11-26 11:00:29 UTC
This failed on RHEV34 too, how do you want to proceed bdunne?

Comment 25 Brandon Dunne 2015-11-30 16:51:25 UTC
Pete,  Can you provide the CloudInit Customization Template that you are using?  In my testing, everything appears to be fine on RHEV 3.4 and 3.5.0.

Comment 27 errata-xmlrpc 2015-12-08 13:20:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:2551

Comment 28 Greg McCullough 2016-01-08 19:11:31 UTC
*** Bug 1156592 has been marked as a duplicate of this bug. ***