Bug 1786350 - Cloud init custom script data seem not to get applied on the imported VM
Summary: Cloud init custom script data seem not to get applied on the imported VM
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Virtualization
Version: 2.1.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 2.4.0
Assignee: Jed Lejosne
QA Contact: Ilanit Stein
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-24 15:01 UTC by Ilanit Stein
Modified: 2020-04-20 17:32 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-04-20 17:32:05 UTC
Target Upstream Version:
Embargoed:
apinnick: needinfo-


Attachments (Terms of Use)
cloud-init-in-yaml.png (16.03 KB, image/png)
2019-12-24 15:01 UTC, Ilanit Stein
no flags Details
vm-cirros.yaml (2.27 KB, text/plain)
2020-01-06 14:37 UTC, Marek Libra
no flags Details
vm-fedora.yaml (1023 bytes, text/plain)
2020-01-06 14:38 UTC, Marek Libra
no flags Details

Description Ilanit Stein 2019-12-24 15:01:20 UTC
Created attachment 1647512 [details]
cloud-init-in-yaml.png

Description of problem:
Import a RHEL7.6 VM that has cloud-init package installed, and cloud-init service enabled,
In the Import VM dialog, cloud init screen fill form/custom script.
Both seem to be not implemented as expected on the started VM:

1. Choose Custom script,
enter:
write_files:
-     path: "/tmp/test.txt"  
      content: Here is a line.
Import the VM, and then start the VM.
* In the VM yaml file the cloudInitNoCloud section is in the attached cloud-init-in-yaml.png 
* Result: this file is not created on the VM.

2. Choose to fill cloud-init form, with Hostname and ssh-authorized-keys filled.
Import the VM, and then start the VM.

* This is from the VM yaml file:

        - cloudInitNoCloud:
            userData: |
              #cloud-config
              name: default
              hostname: myhostname
              ssh-authorized-keys:
                - >-
                  ssh-rsa
                  AAAAB3NzaC1yc2EAAAADAQABAAABgQCcZbN8l...ltpXdgdKC8=
                  root
          name: cloudinitdisk

* Result:
- The VM is given the hostname,
- ‘Authenticated SSH Keys’ content, is not in /home/cloud-user/.ssh/authorized_keys. This file exist, but it is empty.

Version-Release number of selected component (if applicable):
CNV-2.2.0-10

Comment 2 Marek Libra 2020-01-06 14:33:13 UTC
I think we are facing two issues here:

Firstly, if I search cloudinit's documentation correctly, the key should be "ssh_authorized_keys" instead of "ssh-authorized-keys".
It means, the kubevirt's documentation [1], kubevirt's example files and openshift/console generators need to be fixed.

Secondly, when trying with cirros VM image, I can see correct content of the "user-data" file on the "cidata" (/dev/vdb) device (means properly copied from the cloudInitNoCloud/userData section of the VM object).
But the boot log shows:
  
   /run/cirros/datasource/data/user-data was not '#!' or executable

Like the '#cloud-config' is not handled properly and an executable is expected only.

In other words, the "cloudInitNoCloud" or "cloudInitConfigDrive" does not work as expected on cirros.

Fabian, can someone confirm proper expected behaviour here?

I am attaching vm-cirros.yaml for that.

[1] https://kubevirt.io/user-guide/docs/latest/creating-virtual-machines/startup-scripts.html

Comment 3 Marek Libra 2020-01-06 14:35:57 UTC
When trying the same thing on fedora image, the "/home/fedora/.ssh/authorized_keys" is populated with the expected key.
I am attaching vm-fedora.yaml for that testing.

Means, just the "ssh_authorized_keys" vs. "ssh-authorized-keys" naming is the issue here.

Comment 4 Marek Libra 2020-01-06 14:37:51 UTC
Created attachment 1650125 [details]
vm-cirros.yaml

Comment 5 Marek Libra 2020-01-06 14:38:27 UTC
Created attachment 1650126 [details]
vm-fedora.yaml

Comment 6 Fabian Deutsch 2020-01-06 14:48:29 UTC
Thanks Marek for analyzing

Stu, can you take the action item of comment 2

Comment 17 Roman Mohr 2020-01-17 21:32:11 UTC
Cirros cloud-init only supports bash. We have tests and they only use bash cloud init workloads. Cirros is a minimal OS without python. I would guess that is the background. We test more complex cloud-init data on Fedora which has python installed. I don't think that we have here a bug.

Comment 19 sgott 2020-01-20 17:56:21 UTC
Closing this bug due to the diagnosis in Comment #17. Please feel free to re-open if you feel this is the wrong resolution.

Comment 20 Ilanit Stein 2020-03-29 13:40:46 UTC
Reopening this bug for the following reasons:

1. The first issue, mentioned in the bug description, on running cloud-init Custom script not working,
was not addressed. 

2. In VM import, cloud-init page, if "ssh-authorized-keys:" is filled, it should automatically set VM yaml with the correct key, "ssh_authorized_keys:", and having it set to "ssh-authorized-keys:" is a virtualization bug, that needs to get fixed. 

@Marek, @Avital,
Regarding the temporary documentation, mentioned in comment #13:
"Marek has approved the following doc change: https://bz1786350_cnv_cloud_init--ocpdocs.netlify.com/openshift-enterprise/latest/cnv/cnv_users_guide/cnv-importing-vmware-vm.html#Importing_vmware_vm_or_template_cnv-importing-vmware-vm

See "Importing a VMware virtual machine or template", step 6."

It says that VM yaml needs to be edited from "ssh-authorized-keys:" into "ssh_authorized_keys:",
and then restart the VM, 
however, at the stage of filling the import dialog, we can not edit the VM yaml. if we use cloud-init, the VM will start after migration with the "ssh-authorized-keys:". The VM can be powered off, and run again, but then I don't think the cloud-init part will take affect.

Comment 21 Avital Pinnick 2020-03-29 13:57:27 UTC
(In reply to Ilanit Stein from comment #20)

> @Marek, @Avital,
> Regarding the temporary documentation, mentioned in comment #13:
> "Marek has approved the following doc change:
> https://bz1786350_cnv_cloud_init--ocpdocs.netlify.com/openshift-enterprise/
> latest/cnv/cnv_users_guide/cnv-importing-vmware-vm.
> html#Importing_vmware_vm_or_template_cnv-importing-vmware-vm
> 
> See "Importing a VMware virtual machine or template", step 6."
> 
> It says that VM yaml needs to be edited from "ssh-authorized-keys:" into
> "ssh_authorized_keys:",
> and then restart the VM, 
> however, at the stage of filling the import dialog, we can not edit the VM
> yaml. 

Correct. The documentation says that the VM yaml must be edited *after* importing: "You must edit the virtual machine’s YAML configuration after importing it:"
The yaml cannot be changed in the import dialog.

It's not the best place to put this workaround, but there was no other good place to put it because the "Import VMware VM" for CVN is a short section.

if we use cloud-init, the VM will start after migration with the
> "ssh-authorized-keys:". The VM can be powered off, and run again, but then I
> don't think the cloud-init part will take affect.

What happens if you stop the VM after import, edit its yaml, and restart it? Are you sure that the cloud-init doesn't work?

Comment 23 sgott 2020-04-03 19:30:50 UTC
(In reply to Ilanit Stein from comment #20)
> Reopening this bug for the following reasons:
> 
> 1. The first issue, mentioned in the bug description, on running cloud-init
> Custom script not working,
> was not addressed. 

To be absolutely clear on this point: the cloud-init data provided in the bug description is a cloud config. This cloud-init format is not supported on Cirros. Cloud config is a yaml file, which requires Python to parse. Python is not present on a default Cirros instance. This was the rationale behind closing the bug in Comment #19 (because only shell script style cloud-init works on this OS). Can you please clarify what you would have expected in terms of addressing this?

> 2. In VM import, cloud-init page, if "ssh-authorized-keys:" is filled, it
> should automatically set VM yaml with the correct key,
> "ssh_authorized_keys:", and having it set to "ssh-authorized-keys:" is a
> virtualization bug, that needs to get fixed. 

True, we did not address this yet.

Comment 24 Ilanit Stein 2020-04-05 14:21:25 UTC
@Stu Gott,

In bug description 2 issues are reported:
1. Import VM using cloud init do not work when filling the cloud init form.
2. Same as 1., but with using custom script.

when I reported the bug I was under the understanding that both issues fail from the same reason.

Now that we have Marek's analysis for issue 1., I understand that issue 1 and issue 2 are separate.
Therefore, I opened or issue 2. Bug 1821024 - Cloud init custom script data seem not to get applied on the imported VM

The issue with using cloud init with Cirros OS, was raised by Marek, and this issue turned out to be expected behavior.

So to sum up, this bug suppose to address issue 1 only,
which is (quoting Marek's input from above): "the key should be "ssh_authorized_keys" instead of "ssh-authorized-keys".

Comment 25 Ilanit Stein 2020-04-20 17:32:05 UTC
Closing this bug since the cloud-init custom script that was used was missing a #cloud-config header.
Import a RHEL-7.6 VM with cloud-init custom script:  

              #cloud-config
              write_files:
              -     path: "/tmp/test.txt"  
                    content: Here is a line.

worked fine: File was created, and content was as expected.


Note You need to log in before you can comment on or make changes to this bug.