Bug 1819960 - NPE on ImportVmTemplateFromConfigurationCommand when creating VM from ovf_data
Summary: NPE on ImportVmTemplateFromConfigurationCommand when creating VM from ovf_data
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.3.8
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ovirt-4.4.1
: 4.4.1
Assignee: Steven Rosenberg
QA Contact: Polina
URL:
Whiteboard:
Depends On: 1830762
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-01 23:13 UTC by Germano Veit Michel
Modified: 2020-09-14 00:04 UTC (History)
11 users (show)

Fixed In Version: ovirt-engine-4.4.1.5
Doc Type: Bug Fix
Doc Text:
Previously, if you used the update template script example of the ovirt-engine-sdk to import a virtual machine or template from an OVF configuration, it failed with a null-pointer exception (NPE). This happened because the script example did not supply the Storage Pool Id and Source Storage Domain Id. The current release fixes this issue. Now, the script gets the correct ID values from the image, so importing a template succeeds.
Clone Of:
: 1830762 (view as bug list)
Environment:
Last Closed: 2020-08-04 13:22:22 UTC
oVirt Team: Virt
Target Upstream Version:
lsvaty: testing_plan_complete-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5394061 0 None None None 2020-09-14 00:04:45 UTC
Red Hat Product Errata RHSA-2020:3247 0 None None None 2020-08-04 13:22:43 UTC
oVirt gerrit 108617 0 ovirt-engine-4.3 MERGED core: Replaced null ids with image ids 2021-01-21 16:04:03 UTC
oVirt gerrit 108778 0 master ABANDONED core: fix import template from ova via the sdk 2021-01-21 16:04:44 UTC
oVirt gerrit 108814 0 master MERGED example: Python3 requires string conversion 2021-01-21 16:04:03 UTC
oVirt gerrit 109511 0 master MERGED core: Fixed upload ova template script error 2021-01-21 16:04:03 UTC
oVirt gerrit 109715 0 ovirt-engine-4.3 MERGED core: Fixed upload ova template script error 2021-01-21 16:04:44 UTC

Description Germano Veit Michel 2020-04-01 23:13:21 UTC
Description of problem:

Importing Vm/Template from Configuration (OVF) fails with NPE when the engine tries to get the size of the disks on the storage

2020-04-02 08:58:11,472+10 INFO  [org.ovirt.engine.core.bll.exportimport.ImportVmTemplateFromConfigurationCommand] (default task-9) [f8d3d7fa-0f0a-4afe-ba72-67bd3d61c158] Running command: ImportVmTemplateFromConfigurationCommand internal: false. Entities affected :  ID: 00000000-0000-0000-0000-000000000000 Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN
2020-04-02 08:58:11,474+10 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (default task-9) [f8d3d7fa-0f0a-4afe-ba72-67bd3d61c158] createCommand failed: null
2020-04-02 08:58:11,475+10 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (default task-9) [f8d3d7fa-0f0a-4afe-ba72-67bd3d61c158] Exception: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.8.0_242]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [rt.jar:1.8.0_242]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.8.0_242]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [rt.jar:1.8.0_242]
        at org.ovirt.engine.core.vdsbroker.ResourceManager.instantiateInjectedCommand(ResourceManager.java:365) [vdsbroker.jar:]
        at org.ovirt.engine.core.vdsbroker.ResourceManager.createCommand(ResourceManager.java:349) [vdsbroker.jar:]
        at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:395) [vdsbroker.jar:]
        at org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown Source) [vdsbroker.jar:]
        at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) [:1.8.0_242]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_242]
        at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_242]
        at org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:]
        at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source) [:1.8.0_242]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_242]
        at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_242]
        at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68) [weld-core-impl.jar:3.0.6.Final-redhat-00003]
        at org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown Source) [vdsbroker.jar:]
        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.ImagesHandler.prepareImage(ImagesHandler.java:992) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.ImagesHandler.getQemuImageInfoFromVdsm(ImagesHandler.java:843) [bll.jar:]
        at org.ovirt.engine.core.bll.exportimport.ImportVmTemplateCommand.getQemuImageInfo(ImportVmTemplateCommand.java:270) [bll.jar:]
        at org.ovirt.engine.core.bll.exportimport.ImportVmTemplateCommand.updateDiskSizeByQcowImageInfo(ImportVmTemplateCommand.java:260) [bll.jar:]
        at org.ovirt.engine.core.bll.exportimport.ImportVmTemplateCommand.initImportClonedTemplateDisks(ImportVmTemplateCommand.java:146) [bll.jar:]
        at org.ovirt.engine.core.bll.exportimport.ImportVmTemplateCommandBase.lambda$executeCommand$1(ImportVmTemplateCommandBase.java:292) [bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202) [utils.jar:]
        at org.ovirt.engine.core.bll.exportimport.ImportVmTemplateCommandBase.executeCommand(ImportVmTemplateCommandBase.java:290) [bll.jar:]
        at org.ovirt.engine.core.bll.exportimport.ImportVmTemplateFromConfigurationCommand.executeCommand(ImportVmTemplateFromConfigurationCommand.java:395) [bll.jar:]


Version-Release number of selected component (if applicable):
4.3.8 + fixes for BZ1789737

How reproducible:
Always

Steps to Reproduce:
1. Get an OVA (I used OVA exported by RHV)
2. Import OVA [1]

Actual results:
Fails

Expected results:
Succeeds

Additional info:
[1] https://raw.githubusercontent.com/oVirt/ovirt-engine-sdk/master/sdk/examples/upload_ova_as_template.py
NOTE: this example did not work well for me as is, needed to modify it a bit. 
But the quickest reproducer I found is the following:
1. Create VM on RHV
2. Export VM as OVA
3. Delete VM and Keep the Disk (to avoid upload)
4. Modify the script adding the ca.pem paths, user, FQDN etc and remove the uploads and keep just the create from ovf data.

# Add the template, the transfered disks will be attached to
# this template:
print("Adding template %s" % template_name)
templates_service = connection.system_service().templates_service()
template = templates_service.add(
    types.Template(
        cluster=types.Cluster(
            name=target_cluster_name,
        ),
        initialization=types.Initialization(
            configuration=types.Configuration(
                type=types.ConfigurationType.OVA,
                data=ovf_str
            )
        ),
    ),
)

Comment 1 Germano Veit Michel 2020-04-01 23:16:46 UTC
Note that with BZ1789737 fixes this only happens when importing from configuration (OVF description), when importing the entire OVA directly by placing the OVA on a hypervisor it now works with those fixes in place.

Comment 20 Steven Rosenberg 2020-05-03 17:30:47 UTC
Note: When testing 4.4, python3 will be required. Please convert the ovf_str [1] to a a valid utf 8 string [2] for the test script Germano provided in Comment 17 to work.


[1] data=ovf_str
[2] data=ovf_str.decode("utf-8")

Comment 31 Sandro Bonazzola 2020-05-18 14:46:34 UTC
Moved to 4.4.1 not being marked as blocker for 4.4.0 and we are preparing to GA.

Comment 32 Arik 2020-06-18 15:49:06 UTC
The missing piece for this to work on 4.4 was separated out to bz 1848586

Comment 33 Sandro Bonazzola 2020-06-19 09:39:36 UTC
Arik, isn't 4.3.11 handled by bug #1830762 ? Shouldn't this be targeted to 4.4.1?

Comment 34 Arik 2020-06-21 07:40:49 UTC
(In reply to Sandro Bonazzola from comment #33)
> Arik, isn't 4.3.11 handled by bug #1830762 ? Shouldn't this be targeted to
> 4.4.1?

Right, changing back

Comment 39 Polina 2020-07-05 11:16:58 UTC
Verification on 
ovirt-engine-4.4.1.2-0.10.el8ev.noarch
libvirt-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64
vdsm-4.40.19-1.el8ev.x86_64

according to description, 1819960#c17 and 1819960#c35

Test1. 

The manual work done:
1. Create VM on RHV
2. Export VM as OVA
3. Delete VM and Keep the Disk (to avoid upload).

The script from 1819960#c17 with changed fqdn and credentials brought the following error:

python3 upload_ova_as_template.py ova_test_no_template.ova golden_env_mixed_1 nfs_0

ovirtsdk4.Error: The CA file '/etc/pki/ovirt-engine/ca.pem' doesn't exist 

Though the file does exists and the path is correct.

After changing the script to ca_file=None, insecure=True, I could continue. 

python3 upload_ova_as_template.py ova_test_no_template.ova golden_env_mixed_1 nfs_0

Connecting...
Adding template ova_test_no_template.

template is successfully added and VMs could be created from this template.

If the same script was run without removing the VM it fails with "[Internal Engine Error]". HTTP response code is 400. As I understand it is known issue . Please confirm 


Test 2. If the same Test1 was performed for a VM that originally was created from template latest-rhel-guest-image-8.2-infra, the scenario doesn't work.

[root@ocelot01 tmp]# python3 upload_ova_as_template.py polina.ova golden_env_mixed_1 nfs_0
Connecting...
Adding template polina
....
ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot import Template. The VM template disk status is not 'OK'.]". HTTP response code is 409.

2020-07-05 13:25:48,894+03 ERROR [org.ovirt.engine.core.bll.exportimport.ImportVmFromConfigurationCommand] (default task-15) [583e39d8-6647-423b-ae9d-608a6f79ce7c] found an image (6fb9e737-83d7-471d-8d20-83c19bf56604) whose status is ILLEGAL
2020-07-05 13:25:48,899+03 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-15) [] Operation Failed: [Cannot import Template. The VM template disk status is not 'OK'.]


Please review both tests and confirm if the bug must be verified with these results.

Comment 40 Arik 2020-07-05 11:51:32 UTC
(In reply to Polina from comment #39)
> ovirtsdk4.Error: The CA file '/etc/pki/ovirt-engine/ca.pem' doesn't exist 
> 
> Though the file does exists and the path is correct.

Do you run the script on the host that the engine runs on?

> If the same script was run without removing the VM it fails with "[Internal
> Engine Error]". HTTP response code is 400. As I understand it is known issue
> . Please confirm 

We're probably missing a validation that checks whether a VM exists when importing the template.
Please file a separate bug and attach logs for this part.

> Test 2. If the same Test1 was performed for a VM that originally was created
> from template latest-rhel-guest-image-8.2-infra, the scenario doesn't work.
> 
> [root@ocelot01 tmp]# python3 upload_ova_as_template.py polina.ova
> golden_env_mixed_1 nfs_0
> Connecting...
> Adding template polina
> ....
> ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is
> "[Cannot import Template. The VM template disk status is not 'OK'.]". HTTP
> response code is 409.
> 
> 2020-07-05 13:25:48,894+03 ERROR
> [org.ovirt.engine.core.bll.exportimport.ImportVmFromConfigurationCommand]
> (default task-15) [583e39d8-6647-423b-ae9d-608a6f79ce7c] found an image
> (6fb9e737-83d7-471d-8d20-83c19bf56604) whose status is ILLEGAL
> 2020-07-05 13:25:48,899+03 ERROR
> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
> task-15) [] Operation Failed: [Cannot import Template. The VM template disk
> status is not 'OK'.]
> 

I'm not able to remove a VM that is based on a template (thin-provisioned) without removing its disks from the webadmin.
Did you do it from the API?

Comment 41 Polina 2020-07-05 13:20:31 UTC
(In reply to Arik from comment #40)

> > ovirtsdk4.Error: The CA file '/etc/pki/ovirt-engine/ca.pem' doesn't exist 
> > Though the file does exists and the path is correct.
> 
> Do you run the script on the host that the engine runs on?

yes. 

> We're probably missing a validation that checks whether a VM exists when
> importing the template.
> Please file a separate bug and attach logs for this part.

ok
 
> > Test 2. If the same Test1 was performed for a VM that originally was created
> > from template latest-rhel-guest-image-8.2-infra, the scenario doesn't work.
> > 
> > [root@ocelot01 tmp]# python3 upload_ova_as_template.py polina.ova
> > golden_env_mixed_1 nfs_0
> > Connecting...
> > Adding template polina
> > ....
> > ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is
> > "[Cannot import Template. The VM template disk status is not 'OK'.]". HTTP
> > response code is 409.
> > 
> > 2020-07-05 13:25:48,894+03 ERROR
> > [org.ovirt.engine.core.bll.exportimport.ImportVmFromConfigurationCommand]
> > (default task-15) [583e39d8-6647-423b-ae9d-608a6f79ce7c] found an image
> > (6fb9e737-83d7-471d-8d20-83c19bf56604) whose status is ILLEGAL
> > 2020-07-05 13:25:48,899+03 ERROR
> > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
> > task-15) [] Operation Failed: [Cannot import Template. The VM template disk
> > status is not 'OK'.]
> > 
> 
> I'm not able to remove a VM that is based on a template (thin-provisioned)
> without removing its disks from the webadmin.
> Did you do it from the API?

No. I don't do it from API.
In UI when removing the VM (no matter if it was created from template or not - behaves the same) uncheck 'Remove Disk(s)' option in 'Remove Virtual Machine(s)' popup window. It removes the VM and leave the disk

Comment 42 Polina 2020-07-05 19:48:15 UTC
since the situation with ILLEGAl image is not reproducible after engine restart and the first issue is inserted as a separate bug (https://bugzilla.redhat.com/show_bug.cgi?id=1853924) I verify this one.

Comment 48 errata-xmlrpc 2020-08-04 13:22:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: RHV Manager (ovirt-engine) 4.4 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3247


Note You need to log in before you can comment on or make changes to this bug.