Bug 2010863

Summary:	Ansible jobs never start
Product:	Red Hat Satellite	Reporter:	Lukáš Hellebrandt <lhellebr>
Component:	Ansible	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED ERRATA	QA Contact:	Lukáš Hellebrandt <lhellebr>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.10.0	CC:	ahumbe, aruzicka, jpathan, mmccune, oezr, pcreech, vsedmik
Target Milestone:	6.10.0	Keywords:	Regression, Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-daemon-2.1.2-7.2	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-11-16 14:13:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lukáš Hellebrandt 2021-10-05 14:06:33 UTC

Description of problem:
In snap 21.0, running an Ansible role results in the following in /var/log/foreman-proxy/proxy.log:

```
2021-10-05T09:35:46  [E] error while dispatching request to runner ff3d243e-8a1a-4748-ac2c-8c6d920f4430:TypeError no implicit conversion of nil into String:
 /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible_core-4.2.0/lib/foreman_ansible_core/runner/ansible_runner.rb:32:in `basename'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible_core-4.2.0/lib/foreman_ansible_core/runner/ansible_runner.rb:32:in `refresh'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-core-0.3.6/lib/foreman_tasks_core/runner/base.rb:21:in `run_refresh'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-core-0.3.6/lib/foreman_tasks_core/runner/dispatcher.rb:40:in `refresh_runner'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-core-0.3.6/lib/foreman_tasks_core/runner/dispatcher.rb:33:in `start_runner'
```

Consequently, the job never starts and is stuck in "running" state with empty output.

Version-Release number of selected component (if applicable):
Sat 6.10.0 snap 21.0. Regression.

How reproducible:
Deterministic

Steps to Reproduce:
1. Have a Satellite with registered host allowing SSH access to the Satellite
2. In Configure -> Roles, import roles
3. In Host edit dialogue, add some Ansible role
4. In Host edit dialogue, in the top right dropdown menu, select "Run Ansible roles"
5. In Monitoring -> Jobs, open the Job, see it's running, open it for the specific host and see the empty output

Actual results:
The job doesn't run, empty output

Expected results:
Role applied, output shown

Comment 1 Lukáš Hellebrandt 2021-10-05 14:07:40 UTC

Perhaps caused by a fix for bug 1605147

Comment 2 Adam Ruzicka 2021-10-05 17:20:58 UTC

Doesn't seem strictly related to BZ#1605147. I stopped the execution after the directory structure was prepared for ansible-runner and then tried executing ansible-runner manually. Although we should somehow handle the failure more gracefully, let's make the happy path work first.

# ansible-runner run $PWD -p playbook.yml && echo $?
Traceback (most recent call last):
  File "/usr/bin/ansible-runner", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3007, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: docutils

Comment 3 Adam Ruzicka 2021-10-05 17:27:54 UTC

Workaround for now is to install python-docutils

Comment 4 Mike McCune 2021-10-05 18:02:14 UTC

see here for more info:

https://bugzilla.redhat.com/show_bug.cgi?id=2010863#c2

Comment 5 Jameer Pathan 2021-10-07 12:08:07 UTC

*** Bug 2011793 has been marked as a duplicate of this bug. ***

Comment 6 Marek Hulan 2021-10-07 13:15:22 UTC

comment 4 probably meant https://bugzilla.redhat.com/show_bug.cgi?id=1998199#c20, we can keep this for verification but effectively it's a dupe of BZ 1998199 IIUC

Comment 7 Mike McCune 2021-10-12 16:36:58 UTC

moving this to MODIFIED in conjunction with the status of 1998199

Comment 8 Mike McCune 2021-10-14 04:41:23 UTC

FYI, tested this with snap 23 and it worked fine

Comment 9 Lukáš Hellebrandt 2021-10-15 14:28:10 UTC

Verified with Sat 6.10.0 snap 23.0.

Tried "Configure Cloud Connector" workflow, it succeeded.
Tried to apply "RedHatInsights.insights-client" role to a host, it succeeded.

Comment 12 errata-xmlrpc 2021-11-16 14:13:58 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4702