Bug 984491 - Exceptions thrown when attempting to end a task successfully result in the task looping forever.
Exceptions thrown when attempting to end a task successfully result in the ta...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.2.0
x86_64 Linux
urgent Severity high
: ---
: 3.3.0
Assigned To: Omer Frenkel
virt
: Triaged
: 984503 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-15 07:32 EDT by Lee Yarwood
Modified: 2015-09-22 09 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-29 09:12:07 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Lee Yarwood 2013-07-15 07:32:17 EDT
Description of problem:

Exceptions thrown when attempting to end a task successfully result in the task looping forever, for example with an import task :

2013-07-06 21:06:15,423 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-3-thread-49) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type ImportVm threw an exception: org.ovirt.engine.core.compat.CompatException: java.io.IOException: Permission denied
        at org.ovirt.engine.core.compat.backendcompat.Path.GetTempFileName(Path.java:23) [engine-compat.jar:]
        at org.ovirt.engine.core.utils.ovf.OvfWriter.<init>(OvfWriter.java:34) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.ovf.OvfVmWriter.<init>(OvfVmWriter.java:27) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.ovf.OvfManager.ExportVm(OvfManager.java:22) [engine-utils.jar:]
        at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:182) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:156) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.ImportVmCommand.UpdateVmImSpm(ImportVmCommand.java:984) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.ImportVmCommand.endVmRelatedOps(ImportVmCommand.java:974) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.ImportVmCommand.EndImportCommand(ImportVmCommand.java:965) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.ImportVmCommand.endSuccessfully(ImportVmCommand.java:924) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.internalEndSuccessfully(CommandBase.java:512) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:459) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.throw vdsBrokerCommand.getVDSReturnValue().getExceptionObject();(CommandBase.java:1374) [engine-bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:405) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:369) [engine-bll.jar:]

# zgrep -in 'EndAction for action type ImportVm threw an exception' var/log/ovirt-engine/engine* | wc -l
22932

Should we retry a sensible number of times before the task the fails?

Version-Release number of selected component (if applicable):
rhevm-3.1.0-50.el6ev.noarch                                 Sun 24 Mar 2013 08:42:37 PM GMT
rhevm-backend-3.1.0-50.el6ev.noarch                         Sun 24 Mar 2013 08:42:30 PM GMT

How reproducible:
Always

Steps to Reproduce:
1. Break /tmp permissions with chmod 1770 etc.
2. Try to import/export, edit the guest in some way that will result in OVF modification etc.
3. Task will loop attempting to end successfully.

Actual results:
Task loops attempting to end successfully.

Expected results:
Task attempts to end successfully a set number of times before failing.

Additional info:
Comment 4 Omer Frenkel 2013-07-28 04:58:45 EDT
Both problems reported here were solved in 3.3:
1. IO error "Permission denied" when creating temp file:
The code handling XMLs changed in 3.3 and this cannot happen anymore, since it's not using temp file anymore.
this was fixed in http://gerrit.ovirt.org/#/c/11221

2. failure when ending Import-Vm command:
The mechanism tries to rerun end logic in case of failure, and when we encounter endless retries we try to have a specific solution (move error-proned code from end logic, or stop retry for specific errors..)
in this case the error-prone code (which is update the SPM with the new imported vm OVF) was moved already in 3.2 to one general place (OvfDataUpdater) that execute this update out of the end logic, so this error should not be seen anymore in 3.2
this was fixed in http://gerrit.ovirt.org/#/c/9340
Comment 5 Michal Skrivanek 2013-07-29 09:09:53 EDT
*** Bug 984503 has been marked as a duplicate of this bug. ***
Comment 6 Michal Skrivanek 2013-07-29 09:12:07 EDT
the specific case is fixed in 3.2, complete fix for other flows in 3.3

Note You need to log in before you can comment on or make changes to this bug.