Bug 2080718 - NPE while trying to remove a VM during ongoing backup
Summary: NPE while trying to remove a VM during ongoing backup
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.5.0.5
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.5.1
: ---
Assignee: Benny Zlotnik
QA Contact: Evelina Shames
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-01 08:53 UTC by Evelina Shames
Modified: 2022-07-14 05:22 UTC (History)
4 users (show)

Fixed In Version: ovirt-engine-4.5.1.2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-23 05:54:58 UTC
oVirt Team: Storage
Embargoed:
pm-rhel: ovirt-4.5?


Attachments (Terms of Use)
logs (632.73 KB, application/zip)
2022-05-01 08:53 UTC, Evelina Shames
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github oVirt ovirt-engine pull 375 0 None open core: block VM removal during backup 2022-05-15 16:39:22 UTC
Red Hat Issue Tracker RHV-45893 0 None None None 2022-05-01 09:08:16 UTC

Description Evelina Shames 2022-05-01 08:53:48 UTC
Created attachment 1876316 [details]
logs

Description of problem:
As part of our automation tests, one TC failed because it couldn't get this action link:
/ovirt-engine/api/vms/<vm_id>/backups/<backup_id>/finalize

The backup starts:
022-04-29 08:38:22,146+03 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-11) [backups_create_c282888a-5ecd-4997] EVENT_ID: USER_CREATE_SNAPSHOT(45), Snapshot 'Auto-generated for Backup VM' creation for VM 'vm_TestCase27319_2908344326' was initiated by admin@internal-authz.

2022-04-29 08:38:22,209+03 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-11) [backups_create_c282888a-5ecd-4997] EVENT_ID: VM_BACKUP_STARTED(10,790), Backup cd85119f-ae93-4c8a-8e80-07325cfa60ac for VM vm_TestCase27319_2908344326 started (User: admin@internal-authz).

2022-04-29 08:38:37,820+03 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-31) [backups_create_c282888a-5ecd-4997] EVENT_ID: USER_CREATE_SNAPSHOT_FINISHED_SUCCESS(68), Snapshot 'Auto-generated for Backup VM' creation for VM 'vm_TestCase27319_2908344326' has been completed.

Then, it tries to get this action link /ovirt-engine/api/vms/<vm_id>/backups/<backup_id>/finalize to run finalization and failed because it couldn't find it.

Because the test failed, we clean the env before moving to the next test and as part of this step we remove the VM:

2022-04-29 08:38:49,186+03 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-11) [vms_delete_574c271d-c2ce-4a1f] EVENT_ID: USER_REMOVE_VM_FINISHED(113), VM vm_TestCase27319_2908344326 was successfully removed by admin@internal-authz.

Then I start seeing NPE errors:
2022-04-29
 08:38:51,982+03 ERROR [org.ovirt.engine.core.bll.CommandsFactory] 
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-57)
 [backups_create_c282888a-5ecd-4997] An exception has occurred while 
trying to create a command object for command 'CreateSnapshotForVm' with
 parameters 
'CreateSnapshotForVmParameters:{commandId='0f77d597-7821-49ac-9e6d-ec15a9ae0dd7',
 user='admin', commandType='CreateSnapshotForVm', 
vmId='d72a44c7-ef3b-488a-b4c6-af0d7a96e097'}': WELD-000049: Unable to 
invoke protected final void 
org.ovirt.engine.core.bll.CommandBase.postConstruct() on 
org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand@63906c76
2022-04-29
 08:38:51,983+03 ERROR [org.ovirt.engine.core.bll.CommandsFactory] 
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-57)
 [backups_create_c282888a-5ecd-4997] Exception: 
java.lang.NullPointerException
    at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand.isMemorySnapshotSupported(CreateSnapshotForVmCommand.java:702)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand.init(CreateSnapshotForVmCommand.java:129)
    at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.postConstruct(CommandBase.java:217)
    at jdk.internal.reflect.GeneratedMethodAccessor224.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 
   at 
org.jboss.weld.core.6.Final-redhat-00001//org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:83)
 
   at 
org.jboss.weld.core.6.Final-redhat-00001//org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:66)
 
   at 
org.jboss.weld.core.6.Final-redhat-00001//org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:122)
    at deployment.engine.ear//org.ovirt.engine.core.di.Injector.injectMembers(Injector.java:41)
    at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandsFactory.createCommand(CommandsFactory.java:107)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandsRepository.retrieveCommand(CommandsRepository.java:167)
  at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandsRepository.retrieveCommand(CommandsRepository.java:154)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.retrieveCommand(CommandCoordinatorImpl.java:127)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil.retrieveCommand(CommandCoordinatorUtil.java:310)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.getCommand(ChildCommandsCallbackBase.java:189)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:45)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175)
 
   at 
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
 
   at 
org.glassfish.javax.enterprise.concurrent.1.redhat-00001//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360)
 
   at 
org.glassfish.javax.enterprise.concurrent.1.redhat-00001//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
 
   at 
org.glassfish.javax.enterprise.concurrent.1.redhat-00001//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)

2022-04-29
 08:38:51,984+03 ERROR 
[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] 
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-57)
 [backups_create_c282888a-5ecd-4997] Command 'HybridBackup' id: 
'cd85119f-ae93-4c8a-8e80-07325cfa60ac' with children 
[0f77d597-7821-49ac-9e6d-ec15a9ae0dd7] failed when attempting to perform
 the next operation, marking as 'ACTIVE'

Version-Release number of selected component (if applicable):
ovirt-engine-4.5.0.5-0.7.el8ev

How reproducible:
once, couldn't reproduce so far

Steps to Reproduce:
- For the missing action link, I don;t have a reproducer
- For the NPE, try to remove a VM during a running backup.

Actual results:
actions link was missing, NPE when removing a VM during a running backup

Expected results:
action link exists, no NPE in logs, 

Additional info:
Attaching logs

Comment 2 Evelina Shames 2022-05-01 11:23:57 UTC
A new bug for the issue of the missing actions: bug 2080728

Comment 3 Avihai 2022-05-12 11:22:05 UTC
18 Tier2(RHV-4.5-tier2 #10) tests failed on this one, marking as automation blocker

Comment 4 Arik 2022-05-12 11:25:09 UTC
but is it again a consequence of another issue?

Comment 5 Evelina Shames 2022-05-15 07:55:40 UTC
(In reply to Avihai from comment #3)
> 18 Tier2(RHV-4.5-tier2 #10) tests failed on this one, marking as automation
> blocker
these TCs failed because of bug 2080728, removing the automation blocker

Comment 6 Avihai 2022-05-15 11:15:21 UTC
(In reply to Arik from comment #4)
> but is it again a consequence of another issue?
Answered by Evelina in comment#5

Comment 7 Evelina Shames 2022-06-19 12:55:40 UTC
Removing VM while a backup operation is running is now blocked as expected.

Moving to 'Verified'.

Version: ovrit-engine-4.5.1.2-0.11.el8ev(SP1)

Comment 8 Sandro Bonazzola 2022-06-23 05:54:58 UTC
This bugzilla is included in oVirt 4.5.1 release, published on June 22nd 2022.
Since the problem described in this bug report should be resolved in oVirt 4.5.1 release, it has been closed with a resolution of CURRENT RELEASE.
If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.