Red Hat Bugzilla – Bug 984491
Exceptions thrown when attempting to end a task successfully result in the task looping forever.
Last modified: 2015-09-22 09:09 EDT
Description of problem:
Exceptions thrown when attempting to end a task successfully result in the task looping forever, for example with an import task :
2013-07-06 21:06:15,423 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-3-thread-49) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type ImportVm threw an exception: org.ovirt.engine.core.compat.CompatException: java.io.IOException: Permission denied
at org.ovirt.engine.core.compat.backendcompat.Path.GetTempFileName(Path.java:23) [engine-compat.jar:]
at org.ovirt.engine.core.utils.ovf.OvfWriter.<init>(OvfWriter.java:34) [engine-utils.jar:]
at org.ovirt.engine.core.utils.ovf.OvfVmWriter.<init>(OvfVmWriter.java:27) [engine-utils.jar:]
at org.ovirt.engine.core.utils.ovf.OvfManager.ExportVm(OvfManager.java:22) [engine-utils.jar:]
at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:182) [engine-bll.jar:]
at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:156) [engine-bll.jar:]
at org.ovirt.engine.core.bll.ImportVmCommand.UpdateVmImSpm(ImportVmCommand.java:984) [engine-bll.jar:]
at org.ovirt.engine.core.bll.ImportVmCommand.endVmRelatedOps(ImportVmCommand.java:974) [engine-bll.jar:]
at org.ovirt.engine.core.bll.ImportVmCommand.EndImportCommand(ImportVmCommand.java:965) [engine-bll.jar:]
at org.ovirt.engine.core.bll.ImportVmCommand.endSuccessfully(ImportVmCommand.java:924) [engine-bll.jar:]
at org.ovirt.engine.core.bll.CommandBase.internalEndSuccessfully(CommandBase.java:512) [engine-bll.jar:]
at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:459) [engine-bll.jar:]
at org.ovirt.engine.core.bll.CommandBase.throw vdsBrokerCommand.getVDSReturnValue().getExceptionObject();(CommandBase.java:1374) [engine-bll.jar:]
at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:]
at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:]
at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:]
at org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:405) [engine-bll.jar:]
at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:369) [engine-bll.jar:]
# zgrep -in 'EndAction for action type ImportVm threw an exception' var/log/ovirt-engine/engine* | wc -l
Should we retry a sensible number of times before the task the fails?
Version-Release number of selected component (if applicable):
rhevm-3.1.0-50.el6ev.noarch Sun 24 Mar 2013 08:42:37 PM GMT
rhevm-backend-3.1.0-50.el6ev.noarch Sun 24 Mar 2013 08:42:30 PM GMT
Steps to Reproduce:
1. Break /tmp permissions with chmod 1770 etc.
2. Try to import/export, edit the guest in some way that will result in OVF modification etc.
3. Task will loop attempting to end successfully.
Task loops attempting to end successfully.
Task attempts to end successfully a set number of times before failing.
Both problems reported here were solved in 3.3:
1. IO error "Permission denied" when creating temp file:
The code handling XMLs changed in 3.3 and this cannot happen anymore, since it's not using temp file anymore.
this was fixed in http://gerrit.ovirt.org/#/c/11221
2. failure when ending Import-Vm command:
The mechanism tries to rerun end logic in case of failure, and when we encounter endless retries we try to have a specific solution (move error-proned code from end logic, or stop retry for specific errors..)
in this case the error-prone code (which is update the SPM with the new imported vm OVF) was moved already in 3.2 to one general place (OvfDataUpdater) that execute this update out of the end logic, so this error should not be seen anymore in 3.2
this was fixed in http://gerrit.ovirt.org/#/c/9340
*** Bug 984503 has been marked as a duplicate of this bug. ***
the specific case is fixed in 3.2, complete fix for other flows in 3.3