Bug 1776151

Summary: Job result as failure on successful playbook run
Product: Red Hat Satellite Reporter: Ondřej Pražák <oprazak>
Component: Ansible - Configuration ManagementAssignee: Ondřej Pražák <oprazak>
Status: CLOSED ERRATA QA Contact: Lukas Pramuk <lpramuk>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.7.0CC: egolov, inecas, lpramuk, mhulan, rbertolj
Target Milestone: 6.7.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tfm-rubygem-foreman_ansible_core-3.0.2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-04-14 13:27:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1780108    
Bug Blocks:    

Description Ondřej Pražák 2019-11-25 09:02:36 UTC
Even though playbook has no failed tasks, job is marked as failure. This is probably caused by the presence of 'runner_on_failed' event, even though the event should ignore_errors.

<pre>
/tmp/d20191120-17138-1133sp/artifacts/44a932bc-c17c-4615-9806-e5504deed8e7/job_events/20-b917601b-054d-4c5b-8806-597e16c97cff.json:{"event": "runner_on_failed", "uuid": "b917601b-054d-4c5b-8806-597e16c97cff", "stdout": "\u001b[0;31mfatal: [essie-boser.example.com]: FAILED! => {\"changed\": true, \"cmd\": \"test -f /etc/update-motd.d/99-footer\", \"delta\": \"0:00:00.015612\", \"end\": \"2019-11-20 08:07:32.057450\", \"msg\": \"non-zero return code\", \"rc\": 1, \"start\": \"2019-11-20 08:07:32.041838\", \"stderr\": \"\", \"stderr_lines\": [], \"stdout\": \"\", \"stdout_lines\": []}\u001b[0m\r\n\u001b[0;36m...ignoring\u001b[0m", "counter": 20, "pid": 43714, "created": "2019-11-20T08:07:34.838163", "end_line": 196, "runner_ident": "44a932bc-c17c-4615-9806-e5504deed8e7", "start_line": 194, "event_data": {"play_pattern": "all", "play": "all", "event_loop": null, "task_args": "", "uuid": "b917601b-054d-4c5b-8806-597e16c97cff", "remote_addr": "essie-boser.example.com", "res": {"stderr_lines": [], "changed": true, "end": "2019-11-20 08:07:32.057450", "_ansible_no_log": false, "stdout": "", "cmd": "test -f /etc/update-motd.d/99-footer", "start": "2019-11-20 08:07:32.041838", "delta": "0:00:00.015612", "stderr": "", "rc": 1, "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "test -f /etc/update-motd.d/99-footer", "removes": null, "argv": null, "warn": true, "chdir": null, "stdin_add_newline": true, "stdin": null}}, "stdout_lines": [], "msg": "non-zero return code"}, "play_uuid": "b8ca3a6c-3cdc-e22d-694a-000000000006", "task_uuid": "b8ca3a6c-3cdc-e22d-694a-00000000000e", "task": "Check motd tail supported", "playbook_uuid": "2c8d01cc-c3c4-4a7a-b55a-2da53b921fb8", "playbook": "playbook.yml", "task_action": "command", "host": "essie-boser.example.com", "ignore_errors": true, "role": "adriagalin.motd", "task_path": "/etc/ansible/roles/adriagalin.motd/tasks/main.yml:27"}, "parent_uuid": "b8ca3a6c-3cdc-e22d-694a-00000000000e"}

</pre>

Comment 1 Ondřej Pražák 2019-11-25 09:02:40 UTC
Created from redmine issue https://projects.theforeman.org/issues/28313

Comment 4 Ondřej Pražák 2019-11-25 09:08:40 UTC
*** Bug 1776153 has been marked as a duplicate of this bug. ***

Comment 5 Bryan Kearney 2019-11-25 11:06:16 UTC
Upstream bug assigned to oprazak

Comment 6 Bryan Kearney 2019-11-25 11:06:18 UTC
Upstream bug assigned to oprazak

Comment 7 Bryan Kearney 2019-11-25 15:06:52 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/28313 has been resolved.

Comment 8 Brad Buckingham 2019-11-26 14:56:42 UTC
*** Bug 1776829 has been marked as a duplicate of this bug. ***

Comment 9 Brad Buckingham 2019-11-26 14:56:54 UTC
*** Bug 1776828 has been marked as a duplicate of this bug. ***

Comment 10 Lukas Pramuk 2019-11-29 15:26:22 UTC
FailedQA.

1) Install and import adriagalin.motd role

# ansible-galaxy install -p /etc/ansible/roles adriagalin.motd
# hammer ansible roles import --proxy-id 1

2) Assign the role to the Satellite host

# curl -k https://$(hostname):9090/ssh/pubkey >> ~/.ssh/authorized_keys
# hammer host ansible-roles assign --name $(hostname) --ansible-roles adriagalin.motd

3) Play the roles for the host
# hammer host ansible-roles play --name $(hostname)
Ansible roles are being played. Job ID: 6

4) Check the job and its task details

State: running 25% Complete

___/ Run \___

3: Actions::ProxyAction (suspended) [ 988.19s / 0.12s ] Cancel

>>> Job is pending forever

Comment 11 Lukas Pramuk 2019-11-29 15:30:24 UTC
Before the "fix"  Actions::RemoteExecution::RunHostJob during Finalize stage failed

___/ Finalize \___
 4: Actions::RemoteExecution::RunHostJob (error) [ 0.02s / 0.02s ]

Error:
StandardError
Job execution failed

After the "fix" the execution freezes on the previous action Actions::ProxyAction during Run stage

___/ Run \___
 3: Actions::ProxyAction (suspended) [ 988.19s / 0.12s ] Cancel

Comment 12 Marek Hulan 2020-01-08 08:19:52 UTC
I have retested on the last snap and can't reproduce the issue you desribe in comment 11. I tried arbitrary ansible command run and also installed the role according to your setps. Everything seems to work fine. We need your reproducer to unblock this.

Comment 13 Lukas Pramuk 2020-01-20 08:50:27 UTC
I'm no longer able to reproduce comment#11 as well.

The issue reproduced in comment#11 seems to be fixed now and is then unrelated to this BZ.
Switching back to ON_QA then.

Comment 14 Lukas Pramuk 2020-01-20 08:54:35 UTC
VERIFIED.

@satellite-6.7.0-5.beta.el7sat.noarch
tfm-rubygem-foreman_ansible_core-3.0.2-1.el7sat.noarch

by following manual steps: 

1) Install and import adriagalin.motd role

# ansible-galaxy install -p /etc/ansible/roles adriagalin.motd
# hammer ansible roles import --proxy-id 1

2) Assign the role to the Satellite host

# curl -k https://$(hostname):9090/ssh/pubkey >> ~/.ssh/authorized_keys
# hammer host ansible-roles assign --name $(hostname) --ansible-roles adriagalin.motd

3) Play the roles for the host
# hammer host ansible-roles play --name $(hostname)
Ansible roles are being played. Job ID: 6

4) Check the job and its task details

>>> Job finished successfully

Comment 17 errata-xmlrpc 2020-04-14 13:27:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1454