Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1164771

Summary:	Engine-setup should support cleaning of zombie commands before upgrade
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Yair Zaslavsky <yzaslavs>
Component:	ovirt-engine	Assignee:	Simone Tiraboschi <stirabos>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Petr Kubica <pkubica>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	3.5.0	CC:	bazulay, bugs, didi, gklein, lsurette, oourfali, pstehlik, rbalakri, Rhev-m-bugs, sbonazzo, sherold, yeylon, ykaul
Target Milestone:	ovirt-3.6.0-rc	Keywords:	TestCaseNeeded, ZStream
Target Release:	3.6.0	Flags:	stirabos: needinfo-
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Since 3.5 we could also have zombie commands; need to clean also them to avoid getting stuck waiting for task and commands completion as we already do for async task.	Story Points:	---
Clone Of:
Clones:	1196136 (view as bug list)		Environment:
Last Closed:	2016-03-11 07:34:14 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Integration	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1161012
Bug Blocks:

Description Yair Zaslavsky 2014-11-17 12:20:54 UTC

Description of problem:

Similar to tasks, commands have parameters, which may change in structure between two versions of oVirt.
Such chances are problematic, deserialization of the commands may fail after engine startup - this will yieled a bad command execution and an unexpected behavior.

It is required to allow upgrade only if there are no running tasks.

BZ1161012 extends the task cleaning utility, introducing new flags :

-T removes/displays commands with running tasks
-o removes/displays all commands
-r remove all commands (use with -T to clear only those with running tasks).




Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Sandro Bonazzola 2014-11-18 15:01:09 UTC

I see that bug #1161012 has been merged for 3.5.1, can you provide some info about:
1) how to reach a condition where commands / tasks are running for enough time to allow us to start an upgrade
2) how critical is this fix: it's not clear what happen if you upgrade while the command is running, do we have data loss?

Targeting tentatively to 3.5.2, will retarget if needed when more info are available.

Comment 2 Yair Zaslavsky 2014-11-23 13:49:09 UTC

(In reply to Sandro Bonazzola from comment #1)
> I see that bug #1161012 has been merged for 3.5.1, can you provide some info
> about:
> 1) how to reach a condition where commands / tasks are running for enough
> time to allow us to start an upgrade

Create a storage related operation that takes time - let's say creating a preallocated disk with 50 - 100 giga?
Then stop engine (to perform upgrade), and then run engine-upgrade.

> 2) how critical is this fix: it's not clear what happen if you upgrade while
> the command is running, do we have data loss?

Quite critical, as parameters of commands may vary between two versions, and diseralizaing of commands will fail. This may lead to an inconsistent state at engine.

> 
> Targeting tentatively to 3.5.2, will retarget if needed when more info are
> available.

Comment 3 Yedidyah Bar David 2014-12-17 14:00:03 UTC

What do we actually want to do? Just kill them all, like we do with tasks? Ask the user if we find any? Something else?

Comment 4 Simone Tiraboschi 2014-12-17 14:03:36 UTC

We are already using task cleaner utility with -R option which should be CLEAR_ALL.
Isn't it enough to remove all the tasks, all the commands and all the compensations in a single option?

Comment 5 Simone Tiraboschi 2014-12-19 14:12:42 UTC

Sorry, I got it.
We just have to remove the zombie commands (to avoid getting stuck) and wait till the running ones complete.

Comment 6 Eyal Edri 2015-02-25 08:43:42 UTC

3.5.1 is already full with bugs (over 80), and since none of these bugs were added as urgent for 3.5.1 release in the tracker bug, moving to 3.5.2

Comment 7 Sandro Bonazzola 2015-02-25 11:05:25 UTC

Re-targeting to 3.5.1, without this one Bug #1161012 is not fully fixed.

Comment 10 Petr Kubica 2015-04-17 10:11:14 UTC

Verified in ovirt 3.6.0-0.0.master.20150416173208.git88a29e5.el6

upgrade from 3.5.1 to 3.6.0

For testing I changed the value of AsyncTaskZombieTaskLifeInMinutes in table vdc_options from 3000 to 30 minutes.