Bug 1525905

Summary: [RFE] otopi should notify about nonexistent before=/after= events
Product: [oVirt] otopi Reporter: Yedidyah Bar David <didi>
Component: CoreAssignee: Yedidyah Bar David <didi>
Status: CLOSED CURRENTRELEASE QA Contact: Lucie Leistnerova <lleistne>
Severity: low Docs Contact:
Priority: low    
Version: masterCC: bugs, lleistne
Target Milestone: ovirt-4.4.0Keywords: CodeChange, FutureFeature, Reopened
Target Release: 1.9.0Flags: sbonazzo: ovirt-4.4?
rule-engine: planning_ack?
sbonazzo: devel_ack+
lleistne: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: otopi-1.9.0 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-20 20:00:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1700864    

Description Yedidyah Bar David 2017-12-14 11:00:28 UTC
Description of problem:

Currently, if otopi sees an event like:

    @plugin.event(
        stage=plugin.Stages.STAGE_MISC,
        after=(
            'nonexisting_event',
        ),
    )
    def _misc(self):

it simply ignores this 'nonexisting_event'.

It should probably fail.

Once we do this, we are likely to find some bugs in real code, so perhaps at first we should fail optionally for some time/versions.

This came up recently in ovirt-hosted-engine-setup. It had title events, that were used to show titles to the user during customization, but also to keep a partially-strict order of the questions. But only "setup" flow used them, "upgrade-appliance" did not. And no-one noticed. Then we moved things around so that we always used these tittles, and suddenly started getting failures in buildSequence due to cycles.

Comment 1 Yaniv Lavi 2018-06-12 11:15:36 UTC
Closing old RFEs, please reopen if still needed.
Patches are always welcome.

Comment 2 Yedidyah Bar David 2018-06-13 05:35:17 UTC
I think it's important.

(Also not sure what's so old about this bug - less than half a year old).

Comment 3 Yedidyah Bar David 2019-09-02 06:02:13 UTC
otopi currently fails with an error message, if it finds a method that has before=/after= parameters that point to method name where no method has this name. For example:

[ ERROR ] "after" parameter of method "otopi.plugins.ovirt_engine_setup.ovirt_engine_dwh.db.vacuum.Plugin._customization" refers to a method name "osetup.db.connection.status", but no method with this name exists
[ ERROR ] "after" parameter of method "otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaa.Plugin._customization" refers to a method name "osetup.db.connection.status", but no method with this name exists
[ ERROR ] "after" parameter of method "otopi.plugins.ovirt_engine_setup.ovirt_engine.db.vacuum.Plugin._customization" refers to a method name "osetup.db.connection.status", but no method with this name exists
[ ERROR ] "after" parameter of method "otopi.plugins.ovirt_engine_setup.ovirt_engine.fence_kdump_listener.config.Plugin._customization_disable" refers to a method name "osetup.db.connection.status", but no method with this name exists
[ ERROR ] "after" parameter of method "otopi.plugins.ovirt_engine_setup.ovirt_engine.fence_kdump_listener.config.Plugin._customization_firewall" refers to a method name "osetup.db.connection.status", but no method with this name exists
[ ERROR ] Failed to execute stage 'Environment setup': Found bad "before" or "after" parameters

Such errors in existing code where fixed, so under normal circumstances, they should not happen, thus setting doc-text "-".

QE: Nothing for you to test in a positive flow, just that normal runs of existing otopi users (engine-setup, host-deploy) finish successfully. If you want to create a negative flow, you can see as an example the test I added to CI:

https://gerrit.ovirt.org/102205

Comment 4 Yedidyah Bar David 2019-09-11 10:18:23 UTC
Sorry, but this broke engine-cleanup, so disabling by default for now. You can enable by adding to the answer file, or otopi command line:

CORE/ignoreMissingBeforeAfter=bool:False

The fix to make engine-cleanup work is not that simple. A concrete example: In [1] we fail on engine-cleanup, because both odwhcons.Stages.DB_SCHEMA and oengcommcons.Stages.DB_CONNECTION_AVAILABLE run only in setup, but [1] itself runs also in cleanup (it's in common) and we need to keep it this way (for updating the engine db that dwh is being removed). So a fix is probably to move both from setup to common, but this needs to be verified carefully, in several different flows (engine and dwh on same machine, on separate ones, dwh on a separate machine where engine is also installed but not configured, etc.). So decided to postpone for now. Going to merge the added patches soon, to un-break engine-cleanup, and filed bug 1751164 for tracking this issue.

[1] https://github.com/oVirt/ovirt-dwh/blob/master/packaging/setup/plugins/ovirt-engine-common/ovirt-engine-dwh/db/engine_connection.py#L219

Comment 5 Yedidyah Bar David 2019-09-18 10:09:18 UTC
Moving to MODIFIED, as all relevant patches merged. Currently we do not notify, because it's unneeded new noise about existing problems. We do log, though. So if you want to verify current bug, you can run engine-cleanup. It will run as usual, but the log file will contain also:

2019-09-18 13:06:41,159+0300 DEBUG otopi.context context.check:738 "before" parameter of method "otopi.plugins.ovirt_engine_common.base.network.firewall_manager_firewalld.Plugin._setup" refers to a method name "osetup.keep.only.valid.firewall.managers", but no method with this name exists
2019-09-18 13:06:41,160+0300 DEBUG otopi.context context.check:738 "before" parameter of method "otopi.plugins.ovirt_engine_common.base.network.firewall_manager_human.Plugin._setup" refers to a method name "osetup.keep.only.valid.firewall.managers", but no method with this name exists
2019-09-18 13:06:41,160+0300 DEBUG otopi.context context.check:738 "before" parameter of method "otopi.plugins.ovirt_engine_common.base.network.firewall_manager_iptables.Plugin._setup" refers to a method name "osetup.keep.only.valid.firewall.managers", but no method with this name exists
2019-09-18 13:06:41,160+0300 DEBUG otopi.context context.check:738 "before" parameter of method "otopi.plugins.ovirt_engine_common.base.network.hostname.Plugin._customization" refers to a method name "osetup.dialog.titles.network.end", but no method with this name exists
2019-09-18 13:06:41,161+0300 DEBUG otopi.context context.check:738 "after" parameter of method "otopi.plugins.ovirt_engine_common.base.network.hostname.Plugin._customization" refers to a method name "osetup.dialog.titles.network.start", but no method with this name exists
2019-09-18 13:06:41,161+0300 DEBUG otopi.context context.check:738 "before" parameter of method "otopi.plugins.ovirt_engine_common.ovirt_engine.system.he.Plugin._validate" refers to a method name "osetup.memory.check", but no method with this name exists
2019-09-18 13:06:41,161+0300 DEBUG otopi.context context.check:738 "after" parameter of method "otopi.plugins.ovirt_engine_common.ovirt_engine_dwh.db.engine_connection.Plugin._engine_connection" refers to a method name "osetup.dwh.db.schema", but no method with this name exists
2019-09-18 13:06:41,161+0300 DEBUG otopi.context context.check:738 "after" parameter of method "otopi.plugins.ovirt_engine_common.ovirt_engine_dwh.db.engine_connection.Plugin._engine_connection" refers to a method name "osetup.db.connection.available", but no method with this name exists
2019-09-18 13:06:41,161+0300 DEBUG otopi.context context.check:738 "after" parameter of method "otopi.plugins.ovirt_engine_common.ovirt_engine_common.system.apache.Plugin._closeup" refers to a method name "osetup.core.engine.start", but no method with this name exists

Also, otopi's source code includes an example plugin that causes this, which is used to verify current bug in CI.

Comment 6 Sandro Bonazzola 2020-05-20 20:00:37 UTC
This bugzilla is included in oVirt 4.4.0 release, published on May 20th 2020.

Since the problem described in this bug report should be
resolved in oVirt 4.4.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.