Bug 1189813 - provision rhev vms with cloudinit user-data templates
Summary: provision rhev vms with cloudinit user-data templates
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Compute Resources - RHEV
Version: Unspecified
Hardware: Unspecified
OS: Linux
Target Milestone: Unspecified
Assignee: Ivan Necas
QA Contact: Ivan Necas
URL: http://projects.theforeman.org/issues...
: 1480349 (view as bug list)
Depends On:
Blocks: rhci-common-installer 1317530 1479954 1480349
TreeView+ depends on / blocked
Reported: 2015-02-05 14:55 UTC by Karim Boumedhel
Modified: 2021-06-10 10:51 UTC (History)
30 users (show)

Fixed In Version: tfm-rubygem-rbovirt-0.1.4-1.el7sat
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1479954 (view as bug list)
Last Closed: 2018-02-21 16:54:17 UTC
Target Upstream Version:

Attachments (Terms of Use)
Backport of https://github.com/abenari/rbovirt/pull/80/commits/0977712b2082ddebd69cfade8240f5fbe3eb96c9 against rbovirt 0.0.38 (satellite 6.2) (2.39 KB, patch)
2017-06-23 07:20 UTC, Ivan Necas
no flags Details | Diff
Backport of https://github.com/theforeman/foreman/pull/2350/commits/12ffd00fbbb125b7a411961760653c071005aff8 to foreman 1.11 (satellite 6.2) (1.95 KB, patch)
2017-06-23 07:21 UTC, Ivan Necas
no flags Details | Diff
Hotfix packages with additional commit to rbovirt to fix compatibility for cloud-init setup with RHEV 4 (16.98 MB, application/x-gzip)
2017-07-27 14:52 UTC, Ivan Necas
no flags Details
Hotfix RPMs - third version (16.98 MB, application/x-gzip)
2017-07-27 14:55 UTC, Ivan Necas
no flags Details
template (2.69 KB, text/plain)
2017-08-04 13:08 UTC, Daniele
no flags Details
cloud-init output (2.56 KB, text/plain)
2017-08-04 13:09 UTC, Daniele
no flags Details
cloud-init log (240.85 KB, text/plain)
2017-08-04 13:10 UTC, Daniele
no flags Details
cloud.tar.gz (7.63 KB, application/x-gzip)
2017-08-04 13:10 UTC, Daniele
no flags Details
Hotfix RPMs - 4th version (16.98 MB, application/x-gzip)
2017-08-23 14:35 UTC, Ivan Necas
no flags Details

System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 8289 0 Normal Closed use cloudinit user data in ovirt/rhev compute ressource 2020-12-14 17:14:54 UTC
Foreman Issue Tracker 20591 0 Normal Closed Support for multi-line runcmd cloudinit commands in oVirt 2020-12-14 17:14:25 UTC
Red Hat Knowledge Base (Solution) 2221221 0 None None None 2016-11-06 13:38:47 UTC

Description Karim Boumedhel 2015-02-05 14:55:42 UTC
Description of problem:
provision rhev vms with cloudinit user-data templates
we d like to be able to use cloudinit like templates to customize rhev vms we create from satellite6 the upstream 

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
associate a user-data template to a rhev compute ressource

launch a vm based on this template

Actual results:
cant be done in the UI 

Expected results:
vm should be provisioned and started with the cloudinit rendered from the user-data template

Additional info:
pullrequest is at https://github.com/theforeman/foreman/pull/1939

Comment 1 RHEL Program Management 2015-02-05 15:22:55 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 3 Bryan Kearney 2015-04-29 17:50:27 UTC
Connecting redmine issue http://projects.theforeman.org/issues/8289 from this bug

Comment 4 Bryan Kearney 2015-08-25 18:33:18 UTC
Upstream bug component is Compute Resources

Comment 6 John Matthews 2016-02-26 15:07:43 UTC
This is of high value to RHCI.

Comment 11 Bryan Kearney 2016-06-15 08:05:05 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/8289 has been closed

Comment 21 Ivan Necas 2017-06-23 07:20:20 UTC
Created attachment 1290909 [details]
Backport of https://github.com/abenari/rbovirt/pull/80/commits/0977712b2082ddebd69cfade8240f5fbe3eb96c9 against rbovirt 0.0.38 (satellite 6.2)

Comment 22 Ivan Necas 2017-06-23 07:21:39 UTC
Created attachment 1290910 [details]
Backport of https://github.com/theforeman/foreman/pull/2350/commits/12ffd00fbbb125b7a411961760653c071005aff8 to foreman 1.11 (satellite 6.2)

Comment 44 Ivan Necas 2017-07-27 14:52:05 UTC
Created attachment 1305440 [details]
Hotfix packages with additional commit to rbovirt to fix compatibility for cloud-init setup with RHEV 4

There were addition compatibility changes needed for the cloud init provisioning to work with RHEV 4.

I've tested the code with Satellite 6.2.9 and RHEV 4 and worked as expected: the cloud-init data were available in the VM under `/dev/sr1` when the provisioning finished.

Comment 45 Ivan Necas 2017-07-27 14:55:00 UTC
Created attachment 1305442 [details]
Hotfix RPMs - third version

Updated hotfix to work with RHEV4. Marking previous versions as obsolete. Tested against 6.2.9 and RHEV 4.1.2

Comment 47 Daniele 2017-08-04 13:08:19 UTC
Hello Ivan,

This version indeed seems to work a lot better.
The payload seems to get correctly to RHV and the data is passed to the VM.

We still don't get the behavior the customer expects, though. I wonder if this has anything to do with satellite or rather it's a cloud-init bug/limitation.

I'll post the data here so you can maybe point me to the right direction.

The customer is using a template found upstream as the one shipped with Satellite didn't seem to work at all (can't remember exactly the problem, I'd have to go fetch it in old comments). I'll attach the current template they're using and the screenshots they provided from the logs.

It appears that not all tasks are executed. For example, IP is not changed, hostname is not changed (this seems to be a bug in 7.3) and post scripts are not run (they use a script to workaround the IP change problem, but it doesn't seem to work).

I'll attach all the data they attached to the ticket.

Comment 48 Daniele 2017-08-04 13:08:51 UTC
Created attachment 1309048 [details]

Comment 49 Daniele 2017-08-04 13:09:31 UTC
Created attachment 1309049 [details]
cloud-init output

Comment 50 Daniele 2017-08-04 13:10:16 UTC
Created attachment 1309050 [details]
cloud-init log

Comment 51 Daniele 2017-08-04 13:10:44 UTC
Created attachment 1309051 [details]

Comment 52 Ivan Necas 2017-08-07 15:02:04 UTC
Could you share output of `/unattended/user_data?hostname=host.example.com` (replace the hostname with the real name).

In the logs, there is '2017-08-03 16:55:56,670 - cc_write_files.py[DEBUG]: Skipping module named write-files, no/empty 'write_files' key in configuration', which seems to be connected to the writing for /tmp/foreman-userdata.sh file, therefore I'm interested into how the cloud-init rendered looks like.

Comment 54 Ivan Necas 2017-08-11 10:13:28 UTC
I was able to run simple template like this:

- rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

However, when I tried to do something more useful, such as 

<% if rhel_compatible -%>
- |
<%= indent(2) { snippet('epel') } %>
<% end -%>
- |
<%= indent(2) { snippet('remote_execution_ssh_keys') } %>
<% end -%>

it failed to generate proper user_data template on the system, due to wrong
indentation of the resulting file.

I've opened a minimal fix for the issue in rbovirt gem upstream


There are other gotchas in the cloud-init integration, which mean we
currently support just a small subset of the cloud-init template (I will
file another bug against that) that I don't think are
blocking the initial use of it after fixing https://github.com/abenari/rbovirt/pull/120, but mean the user needs to write the cloud-init template that respects this limitations. I will open different issue for that, as fixing this limitation might get more complicated (as the fix goes through 3 different code-bases), which can slow down the delivery of that part.

Comment 55 Ivan Necas 2017-08-14 14:54:00 UTC
Additional bug, describing the limitation of the current cloud-init support is here https://bugzilla.redhat.com/show_bug.cgi?id=1481315

Comment 56 Satellite Program 2017-08-14 16:09:01 UTC
Upstream bug assigned to inecas

Comment 57 Satellite Program 2017-08-14 16:09:11 UTC
Upstream bug assigned to inecas

Comment 58 Ivan Necas 2017-08-23 13:18:46 UTC
Additional fix https://github.com/abenari/rbovirt/pull/120 merged, putting to POST

Comment 60 Ivan Necas 2017-08-23 14:35:47 UTC
Created attachment 1317125 [details]
Hotfix RPMs - 4th version

Yet another hotfix version, including https://github.com/abenari/rbovirt/pull/120 to support multi-line runcmd commands.

Comment 62 Eric Helms 2017-08-29 15:35:32 UTC
Is there an upstream release of rbovirt to be pulled in?

Comment 63 Ivan Necas 2017-08-29 15:53:32 UTC
Set tfm-rubygem-rbovirt-0.1.4-1.el7sat as fixed in field. I've realized I've forgot to open upstream PR for packaging https://github.com/theforeman/foreman-packaging/pull/1782

Comment 64 Ivan Necas 2017-09-11 07:43:04 UTC
*** Bug 1480349 has been marked as a duplicate of this bug. ***

Comment 68 Satellite Program 2018-02-21 16:54:17 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA.
> For information on the advisory, and where to find the updated files, follow the link below.
> If the solution does not work for you, open a new bug report.
> https://access.redhat.com/errata/RHSA-2018:0336

Comment 69 Dhaval 2018-04-18 04:44:14 UTC
Could you please suggest if i need cloud-init installed in my image? also is this works completely? i dont find it's setting up runcmd, manage_resolve_conf and other modules from cloud-init ? i am using satellite 6.2.12 for Vmware image provisioning.

Comment 70 Ivan Necas 2018-04-23 11:23:36 UTC
The specific implementation of the user_data is provider dependent. VmWare is not using the cloud-init standard, but they rather export a specific set of keys to customize the provisioning. [1].

The data to be generated for this spec should be defined in YAML Foreman in the user_data template.

[1] - https://pubs.vmware.com/vi3/sdk/ReferenceGuide/vim.vm.customization.Specification.html

Comment 71 Dhaval 2018-05-17 11:21:52 UTC
@inecas, one more question please. when user_data template is executed on Client, what network Ports are communicated from Client to vCentre and Client to Capsule or Client to Satellite Master ?


Comment 72 Ivan Necas 2018-05-18 13:46:19 UTC
The question regarding the communication from Client to vCenter is more suitable for the vSphere product support, as it's outside of Satellite's realm.

For the client -> capsule communication, it's not different from the standard network-based provisioning.

Note You need to log in before you can comment on or make changes to this bug.