Bug 1761012

Summary: [RFE] Ability to generate a report for ansible/remote execution task result.
Product: Red Hat Satellite Reporter: Varatharaja Perumal G <vgunasek>
Component: ReportingAssignee: Marek Hulan <mhulan>
Status: CLOSED ERRATA QA Contact: Sam Bible <sbible>
Severity: medium Docs Contact:
Priority: high    
Version: 6.5.0CC: ahumbe, ajambhul, aruzicka, bkearney, egolov, inecas, lhellebr, mhulan, pcreech, saydas, vsedmik, wpinheir, zhunting
Target Milestone: 6.13.0Keywords: EasyFix, FutureFeature, Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: tfm-rubygem-foreman_remote_execution-4.4.0,foreman-2.5.2.9-1,foreman-2.5.2.13-1 Doc Type: Known Issue
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-03 13:20:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Varatharaja Perumal G 2019-10-11 22:25:51 UTC
1. Proposed title of this feature request

Ability to generate a report for ansible/remote execution task result.

2. Who is the customer behind the request?

Added in private note.

3. What is the nature and description of the request?

IHAC looking for a feature to generate/download a report of ansible/remote execution result after the completion of remote execution job.


4. Why does the customer need this? (List the business requirements here)

Customer is running few commands on a list of servers after the completion of remote job.

5. How would the customer like to achieve this? (List the functional requirements here)

Customer want to generate a Excel report from the result.

6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

yes


7. Is there already an existing RFE upstream or in Red Hat Bugzilla?

no


8. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?

ASAP

9. Is the sales team involved in this request and do they have any additional input?

no

10. List any affected packages or components.

N/A

11. Would the customer be able to assist in testing this functionality if implemented?


yes

Comment 5 Marek Hulan 2019-10-14 07:14:32 UTC
Could you please share more information about what columns the report should contain or how it should be structured? Is on record representing a job or a host? Do they want to have STDOUT/STDERR for a particular host included in the report? Do they want to mix REX and Ansible runs in the same report or should it be optional or even two separate reports? I guess that's clear, but it would be based on our DB, meaning once data gets automatically cleaned, they wouldn't be part of the report.

Comment 6 Varatharaja Perumal G 2019-10-21 17:33:32 UTC
Hello Marek,


 Could you please share more information about what columns the report should contain or how it should be structured?

 - we dont have a specific requirement. we can have it as JOB ID, DATE/TIME, HOST, RESULT (STDOUT)

 Is on record representing a job or a host?

 - Its representing a job.

 Do they want to have STDOUT/STDERR for a particular host included in the report? 

 - Customer is expecting the Result of command output. If we have a option to include STDERR it will be great.

 Do they want to mix REX and Ansible runs in the same report or should it be optional or even two separate reports? 

 - Customer is expecting the report based on the job so we can have a two different reports.



Regards 

Perumal

Comment 8 Bryan Kearney 2019-11-01 15:00:03 UTC
*** Bug 1757739 has been marked as a duplicate of this bug. ***

Comment 9 Marek Hulan 2019-11-01 15:05:29 UTC
Thanks for answers. If the report is per job, why do you want to include job ID in it? That will be the same ID for every line in csv right? But that is a small detail we can figure out later. I think it's now clear enough for us to get started.

Comment 10 Bryan Kearney 2020-12-16 19:26:48 UTC
Upstream bug assigned to dmatoule

Comment 11 Bryan Kearney 2020-12-16 19:28:27 UTC
Upstream bug assigned to dmatoule

Comment 12 Bryan Kearney 2021-04-27 12:03:43 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/15265 has been resolved.

Comment 14 Dominik Matoulek 2021-08-06 13:09:45 UTC
Added link to redmine issue where I solved the problem described by Lukáš.

Comment 17 Lukáš Hellebrandt 2021-10-05 14:29:39 UTC
FailedQA for multiple reasons. Tried with Sat 6.10.0 snap 21.0.

1) Attempting to render with some nonexistent job_id results in file only containing headers. No warning.

2) Attempting to render a running job results in "undefined method `in_time_zone' for nil:NilClass"

3) The template is named "Jobs - Invocation report template", inconsistently with the others.

4) stderr is in "stdout" column together with stdout, "stderr" column is empty:

Have this executable file on the host in /root/runthis.py:
```
#!/usr/bin/python

import sys

print("stdout1\nstdout2\nstdout3")
sys.stderr.write("stderr1\nstderr2\nstderr3\n")
```

On the host, run the following SSH job:
```
/root/runthis.py
```

Generate a report and see everything in the stdout column

Note: can't test with Ansible due to bug 2010863.

Comment 18 Marek Hulan 2021-10-12 15:38:47 UTC
I'm under the impressions these were not introduced with the new version right? Considering we're now accepting only blocker fixes, is some of the findings blocker in your opinion?

My assumption
1) not a blocker, should be tracked as a separate bug
2) seems like an edge case, but could be simple to solve
3) imho not a blocker, also not sure what inconsistency you mean, perhaps singular vs plural of the first word? Or the word template? Probably trivial to change
4) I believe this is unrelated bug that was found in REX some time ago, we no longer are able to distinguish between stdout and stderr, I think Adam has more info on this one, perhaps there's already a BZ tracking it

Comment 19 Adam Ruzicka 2021-10-13 09:17:54 UTC
> 4) I believe this is unrelated bug that was found in REX some time ago, we no longer are able to distinguish between stdout and stderr, I think Adam has more info on this one, perhaps there's already a BZ tracking it

The script currently runs in a PTY on the remote side. PTYs provide only one input stream and one output stream, meaning we lose the distinction between stdout and stderr. This is nothing new, this behavior has been there for ages. There are some bits and pieces which can still end up in what Satellite thinks is stderr, but those will be about things that happen before the script is actually started on the other side. Afaik there's no BZ for this yet, but it will most likely go away once we move away from net/ssh

Comment 21 Lukáš Hellebrandt 2021-10-13 09:21:47 UTC
Re comment 19, if we can't distinguish stdout from stderr, that's fine but the report shouldn't contain the stderr column in that case. Except maybe if we plan to fix it and it's for forward compatibility but it should be documented that it doesn't actually contain stderr yet and stdout does.

Comment 26 Marek Hulan 2022-02-04 15:02:02 UTC
The newly linked RM issue resolves 2) and 3). For 1) and 4) recommend opening a separate bugs.

Comment 27 Bryan Kearney 2022-02-05 02:14:26 UTC
Upstream bug assigned to mhulan

Comment 28 Bryan Kearney 2022-02-05 02:14:28 UTC
Upstream bug assigned to mhulan

Comment 29 Marek Hulan 2022-03-28 12:13:19 UTC
*** Bug 1997742 has been marked as a duplicate of this bug. ***

Comment 30 Marek Hulan 2022-05-25 08:44:53 UTC
The RM issue is merged, moving to POST, the fix is in Foreman 3.4

Comment 35 Sam Bible 2023-01-30 20:42:32 UTC
Tested with 6.13 Snap 8

Steps to Verify:
1. Register a Host on Satellite
2. Run some jobs on the host, for example "Run Ansible Roles"
3. Check the job run out on the Jobs page
3. Click the "Generate Report" button, and have it download to the browser.

Results:
CSV file was downloaded properly, and the various columns on the sheet did contain the appropriate information, split accordingly. YAML, JSON and HTML also all work appropriately.

Setting the download at a later date also functions properly.

Personally, it would be nice to see the columns sorted to have the information at the top, rather than the bottom, as when the STDOUT is long ( which it always will be ) the other columns are comparatively difficult to 
view, without some formatting. Doesn't seem to be a blocker to me though, as the information can be imported into any given Excel-ish program, and manipulated.

With respect to email function, mail is delivered appropriately to the inbox (Testing in this case was done using the Host VM as the target, and the mail is delivered appropriately as verified using /var/log/maillog, and visible in /var/mail/root)

Comment 39 errata-xmlrpc 2023-05-03 13:20:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097