Bug 1268137

Summary: Stuck async tasks preventing upgrade (engine-setup) being run
Product: [oVirt] ovirt-engine Reporter: Darryl Bond <darryl.bond>
Component: Setup.EngineAssignee: Simone Tiraboschi <stirabos>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Pavel Stehlik <pstehlik>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: bugs, darryl.bond, didi, lveyde, masayag, oourfali, rmartins, sbonazzo, stirabos, ylavi
Target Milestone: ovirt-3.6.1Flags: ylavi: ovirt-3.6.z?
ylavi: ovirt-4.0.0?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: integration
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-27 12:53:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Collected sos file without var log
none
sos file - var log none

Description Darryl Bond 2015-10-01 22:09:27 UTC
Description of problem:
Trying to transition from 3.6 beta 7 to 1st release candidate.
Run engine-setup to pull the new packages and reconfigure.

Engine-setup stops the engine then complains about 2 async tasks
[ INFO  ] Cleaning async tasks and compensations
          The following system tasks have been found running in the system:
          The following commands have been found running in the system:
          Command ID:           ccbf659f-0124-4563-ac15-7329c1bc8ad0
          Command Type:                                   3203
          Created at:           30
          Status:               ACTIVE                        
          Command ID:           fe98cc6d-edf8-487b-852e-30bbcd4b2be4
          Command Type:                                   3201
          Created at:           30
          Status:               ACTIVE                        
          The following compensations have been found running in the system:
          Would you like to try to wait for that?

Yes waits forever, No exits with the engine stopped.

There is no row in the async_tasks table and vdsClient getTaskStatus show nothing.

There are no outstanding tasks in the GUI.

I set the zombie task timeout to 30 a couple of days ago and attempted the engine-setup upgrade _many_ times and rebooted the engine after each attempt.

I'm confident these tasks are rubbish. How do I get rid of them to perform the upgrade?

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Yaniv Lavi 2015-10-20 11:07:11 UTC
Can you please attach the logs and maybe the db dump as well?

Comment 2 Darryl Bond 2015-10-20 21:25:48 UTC
I tried it regularly for a week or so with the same result.
After the 2nd release candidate was released, I did a yum update on the engine host and tried again. It was successful.
It found the 2 stuck tasks, cleared them and continued with the upgrade.

I can see the logs from when it was unsuccessful but the database dump would likely be useless.

Still want them?

Comment 3 Sandro Bonazzola 2015-10-21 07:44:26 UTC
(In reply to Darryl Bond from comment #2)
> Still want them?

Probably the Db will be useless now that the upgrade succeeded.
Can you please attach the report generated by:
 engine-log-collector --no-hypervisors collect

Comment 4 Darryl Bond 2015-10-21 21:11:27 UTC
Created attachment 1085332 [details]
Collected sos file without var log

Comment 5 Darryl Bond 2015-10-21 21:15:26 UTC
Created attachment 1085333 [details]
sos file - var log

Had to split the file up as it was slightly too big.
Perhaps the collector script should do that.