Bug 1464518 - raw_stdout member of Ansible Tower job object is set to "stdout capture is missing" when using HA Tower
Summary: raw_stdout member of Ansible Tower job object is set to "stdout capture is mi...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Automate
Version: 5.7.0
Hardware: All
OS: All
medium
low
Target Milestone: GA
: cfme-future
Assignee: Bill Wei
QA Contact: Dave Johnson
URL:
Whiteboard: ansible
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-23 16:00 UTC by Samson Wick
Modified: 2019-08-06 20:06 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-20 20:47:21 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Samson Wick 2017-06-23 16:00:23 UTC
Description of problem:
When using Ansible Tower in an HA configuration, the raw_stdout member of the job object is set to "stdout capture is missing" if job refresh was performed against any tower server other than the one where the job was run.

Version-Release number of selected component (if applicable):


How reproducible:

Any time an Ansible Tower Job is executed from Automate against an Ansible Tower management provider in an HA mode, there is a proportional liklihood that the output of the job's raw_stdout member will be erroneously set to "stdout capture is missing".



If the job refresh request was run 

Steps to Reproduce:
1.  Launch an Ansible Tower Job Template from an Automate workflow, record the ansible job ID.

2.  From a method within Automate OR the CFME rails console, reference the job object with the job ID recorded in step 1.


JOB_CLASS = 'ManageIQ_Providers_AnsibleTower_ConfigurationManager_Job'.freeze 
job = $evm.vmdb(JOB_CLASS).find(job_id)

3. Attempt to reference the job object's raw_stdout method

puts job.raw_stdout


Actual results: 

If the most recent refresh was performed against a non-executing Tower server (i.e. any server in the HA configuration other than the one that executed the job) the output will be set to "stdout capture is missing".


Expected results:

job.raw_stdout contains the actual raw STDOUT from the Tower job.


Additional info:

This behavior seems to be caused by the way the ansible tower gem being used by CFME executes the REST call to obtain stdout from the job.  The stdout for jobs is evidently not part of the data stored in the celery db, and instead resides solely on the executing server.  To account for this the Ansible Tower Rest API offers a format option to the request which forces the Tower server receiving the REST request to copy the stdout from the executing Tower server in order to provide it.

By changing the format argument from "txt" to "txt_download" the REST call to Tower consistently provides the correct stdout, however there does not seem to be an obvious way to alter the behavior of CFME to change the format option for the REST call to Ansible Tower.

Here is an illustration using CURL.

Assuming an Tower Job ID of 547.  (For reference, the job ID used internally by CFME and the one used by Tower are different - this example uses the latter).

Unreliable:


curl -f -k -H 'Content-Type: application/json' -XGET --user username:password https://ansible.acme.com/api/v1/jobs/547/stdout/?format=txt


Reliable:


curl -f -k -H 'Content-Type: application/json' -XGET --user username:password https://ansible.acme.com/api/v1/jobs/547/stdout/?format=txt_download

Comment 2 Dustin Scott 2017-07-03 13:23:14 UTC
FYI, the latest version of the ansible-tower-client rubygem offers this functionality:

https://github.com/ansible/ansible_tower_client_ruby/blob/master/lib/ansible_tower_client/base_models/job.rb


    def stdout(format = 'txt')
      out_url = related['stdout']
      return unless out_url
      api.get("#{out_url}?format=#{format}").body
    end


Would need to modify the format variable to utilize txt_download.

Comment 3 Bill Wei 2017-11-20 20:47:21 UTC
With newer version (5.8 or newer) you can get txt_download format from rails console

job.raw_stdout('txt_download')

Comment 4 Samson Wick 2017-11-25 13:05:45 UTC
(In reply to Bill Wei from comment #3)
> With newer version (5.8 or newer) you can get txt_download format from rails
> console
> 
> job.raw_stdout('txt_download')

Is this functionality also available within Automate?

Comment 5 Bill Wei 2017-11-27 20:25:29 UTC
Yes, it is.


Note You need to log in before you can comment on or make changes to this bug.