Bug 1080752

I don't see how this relates to cloud-init.
this is basic template/vm issue, where we need the VM to get a new mac, using normal dhcp as well.

see:
1. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.3/html/Evaluation_Guide/Sealing_a_Linux_template.html

2. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.3/html/Evaluation_Guide/Sealing_a_Linux_Virtual_Machine_for_Deployment_as_a_Template_using_sys-unconfig.html

3. to reduce the workarounds, GSS should re-open Bug 750833 - running sys-unconfig doesn't clean the existing MAC address in the ifcfg-eth0 file.
while virt-sysprep covers it, its not integrated in the flow of vm provisioniing, not easily useful in fast cloning from storage, etc.

Shahar - this isn't a cloud-init problem iiuc. its template sealing guidelines for network aspects.

I think the problem is that 

(In reply to Itamar Heim from comment #5)
> I don't see how this relates to cloud-init.
> this is basic template/vm issue, where we need the VM to get a new mac,
> using normal dhcp as well.
> 
> see:
> 1.
> https://access.redhat.com/site/documentation/en-US/
> Red_Hat_Enterprise_Virtualization/3.3/html/Evaluation_Guide/
> Sealing_a_Linux_template.html
> 
> 2.
> https://access.redhat.com/site/documentation/en-US/
> Red_Hat_Enterprise_Virtualization/3.3/html/Evaluation_Guide/
> Sealing_a_Linux_Virtual_Machine_for_Deployment_as_a_Template_using_sys-
> unconfig.html
> 

The problem is that when cloning a VM from a snapshot you can't do the sealing process.
unless you -
1. clone the VM 
2. Do the sealing 
3. Create a template from the sealed VM
4. Create additional VM

> 3. to reduce the workarounds, GSS should re-open Bug 750833 - running
> sys-unconfig doesn't clean the existing MAC address in the ifcfg-eth0 file.
> while virt-sysprep covers it, its not integrated in the flow of vm
> provisioniing, not easily useful in fast cloning from storage, etc.

Using a template requires sealing it first.
so they have two alternatives:
1. "seal"/prepare the VM before converting it to a tempalte

2. do post-boot fixing via cloud-init, since they can pass any script they want to cloud-init (today) via the API, so they can remove the network config and re-run network detection post boot as well.

but i think they should prepare the VM to become a tempalte. sys-unconfig is not only about the mac address, also ssh fingerprint iirc, etc.

wouldn't it help to clear the network device PCI address?
shouldn't the OS identify it as a new hardware and will have to do some processing and update the mac address?

(In reply to Itamar Heim from comment #8)
> Using a template requires sealing it first.

As Sadique wrote above the customer does not need a template he needs a 'clean' VM from a snapshot, but we do not have a decent way to clean the internal guest configuration.

The only reason I mentioned the template is to emphasis how many steps are needed to create a 'clean' VM - but now thinking of it one can do the sealing and reboot the VM you don't really need to create the template.

> so they have two alternatives:
> 1. "seal"/prepare the VM before converting it to a tempalte
> 
> 2. do post-boot fixing via cloud-init, since they can pass any script they
> want to cloud-init (today) via the API, so they can remove the network
> config and re-run network detection post boot as well.

That was my suggestion but it is not perfect as I think that after cleaning the udev rules you need to reboot the guest for the new rules to be updated.

> 
> but i think they should prepare the VM to become a tempalte. sys-unconfig is
> not only about the mac address, also ssh fingerprint iirc, etc.

(In reply to Omer Frenkel from comment #10)
> wouldn't it help to clear the network device PCI address?
> shouldn't the OS identify it as a new hardware and will have to do some
> processing and update the mac address?

The UDEV rules are not impacted by the PCI addresses so cleaning the PCI addresses won't help us.

Itamar, the scenario in comment #15 looks like a valid use case to me to clone snapshot and run vms from it without making it a template.

Is there a way to get this fixed via a script or in cloud-init?

(In reply to Sadique Puthen from comment #16)
> Itamar, the scenario in comment #15 looks like a valid use case to me to
> clone snapshot and run vms from it without making it a template.
> 
> Is there a way to get this fixed via a script or in cloud-init?

livnat - any update on how others are doing this? i.e., so a cloud-init script will clean/detect the network config post boot?

(In reply to Itamar Heim from comment #17)
> (In reply to Sadique Puthen from comment #16)
> > Itamar, the scenario in comment #15 looks like a valid use case to me to
> > clone snapshot and run vms from it without making it a template.
> > 
> > Is there a way to get this fixed via a script or in cloud-init?
> 
> livnat - any update on how others are doing this? i.e., so a cloud-init
> script will clean/detect the network config post boot?

How about using virt-sysprep on the image of the cloned VM before booting the VM?

1. clone VM from snapshot
2. run virt sysprep on the cloned image
3. boot the VM

Sachin, would you make this bz public? We want users and other customers to be able to find common problems easily.

I'd love to expose virt-sysprep as a Vdsm verb, to be run (optionally) after clone, import, or template creation. Despite the understandable urgency we would be unable to introduce such a change in rhev-3.4.0.

May I suggest to solve this by a taylor-made before_vm_start hook script?
1. clone VM from snapshot
2. add a sysprep_once custom property to the cloned VM
3. start the VM.
4. The hook kicks in, and
   - checks if /var/lib/ovirt/sysprep/<vmid> exists. if so, it finishes.
   - calls virt-sysprep on the VM image
   - touches /var/lib/ovirt/sysprep/<vmid> so next runs won't harm the image
5. remove the sysprep_once custom property. If we trust the use to not forget this step, we can skip creating /var/lib/ovirt/sysprep/<vmid>.

> May I suggest to solve this by a taylor-made before_vm_start hook script?
> 1. clone VM from snapshot
> 2. add a sysprep_once custom property to the cloned VM
> 3. start the VM.
> 4. The hook kicks in, and
>    - checks if /var/lib/ovirt/sysprep/<vmid> exists. if so, it finishes.
>    - calls virt-sysprep on the VM image
>    - touches /var/lib/ovirt/sysprep/<vmid> so next runs won't harm the image
> 5. remove the sysprep_once custom property. If we trust the use to not
> forget this step, we can skip creating /var/lib/ovirt/sysprep/<vmid>.

Dan -
In the suggested solution above if the user uses /var/lib/ovirt/sysprep/<vmid> and the VM would boot the second time on a different host will the image be sealed again?

(In reply to lpeer from comment #20)
I guess the answer is yes:/ 
Since we're touching the image anyway we can perhaps as well touch a file inside the image and check for that (the check itself would become more expensive though)

(In reply to lpeer from comment #20)


> > 4. The hook kicks in, and
> >    - checks if /var/lib/ovirt/sysprep/<vmid> exists. if so, it finishes.
> >    - calls virt-sysprep on the VM image
> >    - touches /var/lib/ovirt/sysprep/<vmid> so next runs won't harm the image
> > 5. remove the sysprep_once custom property. If we trust the use to not
> > forget this step, we can skip creating /var/lib/ovirt/sysprep/<vmid>.
> 
> Dan -
> In the suggested solution above if the user uses
> /var/lib/ovirt/sysprep/<vmid> and the VM would boot the second time on a
> different host will the image be sealed again?

Sorry for not being explicit about /var/lib/ovirt/sysprep/<vmid> existing in the guest os (and yes, Michal, it is expensive to touch).

What is your opinion about the suggested approach of comment 19?

In any case, this is not much of a bug, but a long-missing RFE; my question regarding its publicity is doubled.

I've posted (month ago, sorry for not mentioning it here) a draft for a sysprep hook: http://gerrit.ovirt.org/#/c/26886/ .