Bug 1802270 - Cluster Upgrade fails to run, AnsibleServlet fails to start the upgrade ansible playbook
Summary: Cluster Upgrade fails to run, AnsibleServlet fails to start the upgrade ansib...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Backend.Core
Version: 4.4.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.4.0
: ---
Assignee: Martin Necas
QA Contact: Petr Kubica
URL:
Whiteboard:
Depends On: 1807777
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-12 18:30 UTC by Scott Dickerson
Modified: 2023-09-15 00:29 UTC (History)
6 users (show)

Fixed In Version: ansible-runner-1.4.5, ovirt-engine-4.4.0_beta1
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-20 20:02:04 UTC
oVirt Team: Infra
Embargoed:
pm-rhel: ovirt-4.4+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github ansible ansible-runner pull 422 0 None closed fix: extra_vars type in RunnerConfig 2020-12-05 00:00:47 UTC
Github ansible ansible-runner pull 425 0 None closed fix: move creating logic of job_events folder to run 2020-12-05 00:00:47 UTC
Red Hat Issue Tracker RHV-44088 0 None None None 2021-11-25 14:56:28 UTC
oVirt gerrit 107683 0 None MERGED engine: fix NPE when cluster upgrade 2020-12-05 00:00:20 UTC

Description Scott Dickerson 2020-02-12 18:30:20 UTC
Description of problem:

On webadmin, attempting to run a cluster upgrade fails on the engine.


Version-Release number of selected component (if applicable): engine 4.4


How reproducible: 100%


Steps to Reproduce:
1. In webadmin select a Cluster
2. Click 'Upgrade' and page through the Wizard
3. Click the final blue 'Upgrade' button

Actual results: Toast notifiction "Failed to start the cluster upgrade on XXX"


Expected results: Toast notification that the cluster upgrade has started.


Additional info:
The functionality stops working after BZ1759143 was completed.

While the cluster upgrade wizard is part of ovirt-engine-ui-extensions package, it does use the AnsibleServlet to run a playbook that, in turn, invokes the cluster upgrade role.  The servlet is responding error code 500 and no ansible logs are created.

Relevant server log:
2020-02-12 13:25:49,634-05 TRACE [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default task-27) [] Enter AnsibleExecutor::runCommand
2020-02-12 13:25:49,636-05 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default task-27) [] Exception: null
2020-02-12 13:25:49,636-05 DEBUG [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default task-27) [] Exception: : java.lang.NullPointerException
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleRunnerHTTPClient.runPlaybook(AnsibleRunnerHTTPClient.java:103)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor.runCommand(AnsibleExecutor.java:111)
	at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor.runCommand(AnsibleExecutor.java:58)
	at deployment.engine.ear.services.war//org.ovirt.engine.core.services.AnsibleServlet.lambda$doPost$0(AnsibleServlet.java:92)
	at io.undertow.servlet.26.Final//io.undertow.servlet.spec.ServletContextImpl$2.call(ServletContextImpl.java:181)
	at io.undertow.servlet.26.Final//io.undertow.servlet.spec.ServletContextImpl$2.call(ServletContextImpl.java:178)
	at io.undertow.servlet.26.Final//io.undertow.servlet.spec.ServletContextImpl.invokeRunnable(ServletContextImpl.java:1032)
	at io.undertow.servlet.26.Final//io.undertow.servlet.spec.AsyncContextImpl$4.run(AsyncContextImpl.java:319)
	at org.jboss.threads.3.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.3.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
	at org.jboss.threads.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.base/java.lang.Thread.run(Thread.java:834)

2020-02-12 13:25:49,636-05 ERROR [org.ovirt.engine.core.services.AnsibleServlet] (default task-27) [] Error while executing ansible-playbook command.

Comment 3 Martin Necas 2020-03-08 16:30:22 UTC
I'm waiting for mine PRs to get merged.

Comment 4 Martin Necas 2020-03-12 14:54:57 UTC
Waiting for the release of ansible-runner.

Comment 5 Ritesh Chikatwar 2020-03-12 15:13:32 UTC
Any idea that when will ansible-runner will release as it's blocker for me for this bug.no.-1721366

Comment 6 Martin Perina 2020-03-23 09:36:49 UTC
Moving to 4.4.1, we need to wait for ansible-runner-1.4.5 release

Comment 7 Martin Perina 2020-03-26 09:08:06 UTC
Requires ansible-runner 1.4.5 installed to work properly

Comment 8 Sandro Bonazzola 2020-03-30 13:43:29 UTC
This bug is in modified state and targeted to 4.4.1. Can we re-target to 4.4.0 and move to QE?

Comment 9 Sandro Bonazzola 2020-04-09 08:33:31 UTC
ansible-runner 1.4.5 is now available.

Comment 10 Petr Kubica 2020-04-23 23:31:19 UTC
Verified in 4.4.0-0.33.master.el8ev
tested cluster upgrade

also ansible-runner-1.4.5-1.el8ar.noarch located on engine machine

Comment 11 Sandro Bonazzola 2020-05-20 20:02:04 UTC
This bugzilla is included in oVirt 4.4.0 release, published on May 20th 2020.

Since the problem described in this bug report should be
resolved in oVirt 4.4.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 12 Red Hat Bugzilla 2023-09-15 00:29:34 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days


Note You need to log in before you can comment on or make changes to this bug.