Bug 1376008 - [RFE] otopi should be able to display a list of steps in the transaction at startup
Summary: [RFE] otopi should be able to display a list of steps in the transaction at s...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: otopi
Classification: oVirt
Component: RFEs
Version: master
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ovirt-4.2.0
: 1.7.2
Assignee: Yedidyah Bar David
QA Contact: David Necpal
URL:
Whiteboard:
Depends On:
Blocks: 1367457
TreeView+ depends on / blocked
 
Reported: 2016-09-14 12:49 UTC by Ryan Barry
Modified: 2017-12-20 10:54 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-12-20 10:54:50 UTC
oVirt Team: Integration
Embargoed:
rule-engine: ovirt-4.2+
dnecpal: testing_plan_complete-
ylavi: planning_ack+
sbonazzo: devel_ack+
pstehlik: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 81498 0 master POST core: context: expose methodName 2017-09-06 09:00:14 UTC
oVirt gerrit 81499 0 master POST machine_dialog: Add events data 2017-09-06 09:00:21 UTC

Description Ryan Barry 2016-09-14 12:49:28 UTC
Description of problem:
Currently, there is no way to provide an interface to otopi which provides total completion or a progress bar.

otopi (and otopi's machine dialog) should be able to provide a list of possible transaction steps at startup, along with completion as steps are passed/completed, in order to track progress reliably.

otopi is a branching state machine, and this list of transactions may not be reliable (e.g. some choice may add additional steps, or bypass some steps). otopi should also track this, and flag skipped groups/steps as complete, or directly track completion itself (e.g. "%QCompleted 7/10")

Comment 1 Red Hat Bugzilla Rules Engine 2016-09-14 12:49:38 UTC
Bug tickets must have version flags set prior to targeting them to a release. Please ask maintainer to set the correct version flags and only then set the target milestone.

Comment 2 Yedidyah Bar David 2016-09-14 13:23:40 UTC
otopi already logs the sequence of events it's about to run (search for 'SEQUENCE DUMP - BEGIN' and 'SEQUENCE DUMP - END' in the logs), as well as a line per event run (search for 'context._executeMethod.* METHOD ') and whether the event was actually skipped ('context._executeMethod.* condition '). Is this enough? If so, we only need to formalize this in the machine dialog, right?

However, if we want to make such a progress indicator helpful, we should consider also somehow adding inside-event progress indications. For example, in many cases, most of the time spent in 'engine-setup' is in a single method, from otopi's POV - the one that creates/updates the db schema. This one will not be trivial to handle, but we should be prepared with a protocol allowing this once we do. E.g., something like:

%QEventsListStart
%QEventsListEntry STAGE boot METHOD otopi.plugins.ovirt_engine_common.base.core.misc.Plugin._preinit (None)
%QEventsListEntry STAGE boot METHOD otopi.plugins.ovirt_engine_setup.base.core.misc.Plugin._preinit (None)
...
%QEventsListEntry STAGE init METHOD otopi.plugins.otopi.core.config.Plugin._init (otopi.core.config.init)
...
%QEventsListEnd

(None) for those that do not have a 'name=' attached, or ($name) for those that do.

And then:

%QEventStart Stage boot METHOD otopi.plugins.otopi.packagers.dnfpackager.Plugin._boot
%QEventCondition True
%QEventEnd Stage boot METHOD otopi.plugins.otopi.packagers.dnfpackager.Plugin._boot
...
%QEventStart Stage boot METHOD otopi.plugins.otopi.dialog.machine.Plugin._init
%QEventCondition False
%QEventEnd Stage boot METHOD otopi.plugins.otopi.dialog.machine.Plugin._init

And later on, if/when we implement this, something like:

%QEventStart Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema.Plugin._misc
...
%QSubEventsListStart
%QSubEventsListEntry upgrade/03_06_0000_set_version
%QSubEventsListEntry upgrade/03_06_0010_update_haswell_vds_to_new_name
...
%QSubEventsListEnd
%QSubEventStart upgrade/03_06_0000_set_version.sql
%QSubEventEnd upgrade/03_06_0000_set_version.sql
...

BTW, we made the recently-added tags start with '%Q' where 'Q' is short for 'Query', not sure that's relevant here. Perhaps simply '%EventListStart' etc. or something like that. The Java machine dialog parser (what the engine uses for host-deploy) currently ignores everything starting with '**%', so we are free to do what we want.

Comment 3 Ryan Barry 2016-09-14 15:46:40 UTC
Right --

Formalizing these in the machine dialog is the goal (rather than parsing logs).

Using ovirt-hosted-engine-setup as an example (though anything using otopi would work), there are a large number of steps which never appear to users interacting with some otopi frontend (engine-setup, appears similar).

I would imagine that we would only end up consuming customization stage events from cockpit, though presenting them all is probably best.

I think that in-stage progress events can probably be skipped for now, especially if they're difficult to implement. These will end up being a spinner in cockpit while they're occuring, which is probably enough feedback, and I would imagine a spinner would also be enough for other consumers.

%QEventStart Stage $stage $name
%QEventCondition bool
%QEventEnd Stage $stage $name

Is definitely sufficient for this.

Comment 4 Yedidyah Bar David 2017-09-05 20:34:07 UTC
Note: The new items will all have a prefix of '**%', e.g. '**%QEventsListStart', to prevent issues like bug 1348091. See end of comment 2.

Comment 5 Yedidyah Bar David 2017-09-05 20:41:00 UTC
(In reply to Yedidyah Bar David from comment #4)
> Note: The new items will all have a prefix of '**%', e.g.
> '**%QEventsListStart', to prevent issues like bug 1348091. See end of
> comment 2.

Actually, as explained there, I think it will be nicer to drop the 'Q', so e.g. '**%EventsListStart'.

Comment 6 Yedidyah Bar David 2017-09-05 20:44:06 UTC
Also, I think we are going to reuse the bool constants already introduced before QUERY_HIDDEN_TRUE='TRUE' and QUERY_HIDDEN_FALSE='FALSE', unless someone prefers new constants and/or different values ('True' and not 'TRUE').

Comment 7 Pavel Stehlik 2017-09-14 12:00:10 UTC
This is not HostedEngine specific , correct?

Comment 8 Yedidyah Bar David 2017-09-14 12:10:41 UTC
(In reply to Pavel Stehlik from comment #7)
> This is not HostedEngine specific , correct?

Correct, although I am not aware of any other interested party.

To verify, you can run e.g.:

otopi DIALOG/dialect=str:machine

or:

engine-setup --otopi-environment=DIALOG/dialect=str:machine

You should see, in addition to the output before this bug (e.g. in 4.1):

**%EventsListStart
**%EventsListEntry ...
**%EventsListEntry ...
**%EventsListEnd

And then:

**%EventStart ...
# stuff related to that event
**%EventEnd ...

Comment 9 David Necpal 2017-09-22 15:03:28 UTC
Verified on version: 
ovirt-engine-4.2.0-0.0.master.20170921184504.gitfcfc9a7.el7.centos.noarch

Verified based on suggested steps from comment #8

Comment 10 Sandro Bonazzola 2017-12-20 10:54:50 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.