Created attachment 788034 [details]
engine vdsm logs

Ooops, I took the logs from wrong directory. Sorry and thanks for noticing.

It seems like the vmPayload isn't being serialized as it should which causes to error when trying to read the vm configuration - that's the saved specParams in the snapshots ovf, therefore moving to virt.

<SpecParams><vmPayload><file><openstack/latest/meta_data.json>ewogICJsYXVuY2hfaW5kZXgiIDogIjAiLAogICJhdmFpbGFiaWxpdHlfem9uZSIgOiAibm92YSIs
                 | CiAgInV1aWQiIDogIjQyYTAyNWI0LTViMDctNDkzYy04NjNmLTYwZWU3YmMzMGRhNCIsCiAgIm1l
                 | dGEiIDogewogICAgImVzc2VudGlhbCIgOiAiZmFsc2UiLAogICAgInJvbGUiIDogInNlcnZlciIs
                 | CiAgICAiZHNtb2RlIiA6ICJsb2NhbCIKICB9Cn0=
                 | </openstack/latest/meta_data.json><openstack/latest/user_data>b3V0cHV0OgogIGFsbDogJz4+IC92YXIvbG9nL2Nsb3VkLWluaXQtb3V0cHV0LmxvZycKdXNlcjog
                 | cm9vdApydW5jbWQ6Ci0gJ3NlZCAtaSAnJy9eZGF0YXNvdXJjZV9saXN0OiAvZCcnIC9ldGMvY2xv
                 | dWQvY2xvdWQuY2ZnOyBlY2hvICcnZGF0YXNvdXJjZV9saXN0OgogIFsiTm9DbG91ZCIsICJDb25m
                 | aWdEcml2ZSJdJycgPj4gL2V0Yy9jbG91ZC9jbG91ZC5jZmcnCg==
                 | </openstack/latest/user_data></file><volId>config-2</volId></vmPayload></SpecParams>

Exact steps to reproduce are:
 1. Have a running vm that has Operating System set to "Red Hat Enterprise Linux 6.x"
 2. Create a live snapshot of running vm
 3. Check api/vms/{vm:id}/snapshots

looks like there is a problem in the way cd spec params are saved to the snpashot ovf, which doesnt make a valid xml file.

i think this currently happen only if the vm was run once with cloud init and then live snapshot was taken, so i dont see how this is a regression or a test blocker.
is there other way to reproduce this? (i tried the steps from comment 6 , it didn't work if the vm wasnt started with cloud init)

(In reply to Omer Frenkel from comment #7)
> looks like there is a problem in the way cd spec params are saved to the
> snpashot ovf, which doesnt make a valid xml file.
> 
> i think this currently happen only if the vm was run once with cloud init
> and then live snapshot was taken, so i dont see how this is a regression or
> a test blocker.
It's failing live snapshot sanity and live storage migration sanity tests. And it was not failing on is9.
> is there other way to reproduce this? (i tried the steps from comment 6 , it
> didn't work if the vm wasnt started with cloud init)
I didn't find any other way. I don't think we use cloud-init. Vm was started by:
url:/api/vms/47f22238-5832-48ae-b3bc-2a02380c2655/start
body:<action>
    <async>false</async>
    <grace_period>
        <expiry>10</expiry>
    </grace_period>
    <vm>
        <os>
            <boot dev="hd"/>
            <boot dev="network"/>
        </os>
    </vm>
</action>

(In reply to Jakub Libosvar from comment #8)
> I didn't find any other way. I don't think we use cloud-init. Vm was started
> by:
> url:/api/vms/47f22238-5832-48ae-b3bc-2a02380c2655/start
> body:<action>
>     <async>false</async>
>     <grace_period>
>         <expiry>10</expiry>
>     </grace_period>
>     <vm>
>         <os>
>             <boot dev="hd"/>
>             <boot dev="network"/>
>         </os>
>     </vm>
> </action>

couldn't reproduce the issue using this.
can you please verify it happens always? (without cloud init),
if so, can you please check if some payload defined for the vm?
thanks!

Update: this happens when vm has cloud-init payload: when creating live snapshot, all devices spec params are written to the ovf, withing cloud init has the files, and the file paths are inserted as elements, which is wrong because its not a valid xml value.

and cloud init is used when running first time un-initialized linux vm (same as sysprep for windows) which is new behavior.

Verified in rhevm-3.3.0-0.23.master.el6ev (is16).

Verified either with a clean Linux VM and with Run Once VM with cloud-init enabled. In both cases the live snapshots were successfully created and correctly listed via REST API.

Closing - RHEV 3.3 Released

Closing - RHEV 3.3 Released

Closing - RHEV 3.3 Released