Bug 1464518 - raw_stdout member of Ansible Tower job object is set to "stdout capture is missing" when using HA Tower
raw_stdout member of Ansible Tower job object is set to "stdout capture is mi...
Status: CLOSED CURRENTRELEASE
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Automate (Show other bugs)
5.7.0
All All
medium Severity low
: GA
: cfme-future
Assigned To: Bill Wei
Dave Johnson
ansible
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-23 12:00 EDT by Samson Wick
Modified: 2017-11-20 15:47 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-11-20 15:47:21 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Samson Wick 2017-06-23 12:00:23 EDT
Description of problem:
When using Ansible Tower in an HA configuration, the raw_stdout member of the job object is set to "stdout capture is missing" if job refresh was performed against any tower server other than the one where the job was run.

Version-Release number of selected component (if applicable):


How reproducible:

Any time an Ansible Tower Job is executed from Automate against an Ansible Tower management provider in an HA mode, there is a proportional liklihood that the output of the job's raw_stdout member will be erroneously set to "stdout capture is missing".



If the job refresh request was run 

Steps to Reproduce:
1.  Launch an Ansible Tower Job Template from an Automate workflow, record the ansible job ID.

2.  From a method within Automate OR the CFME rails console, reference the job object with the job ID recorded in step 1.


JOB_CLASS = 'ManageIQ_Providers_AnsibleTower_ConfigurationManager_Job'.freeze 
job = $evm.vmdb(JOB_CLASS).find(job_id)

3. Attempt to reference the job object's raw_stdout method

puts job.raw_stdout


Actual results: 

If the most recent refresh was performed against a non-executing Tower server (i.e. any server in the HA configuration other than the one that executed the job) the output will be set to "stdout capture is missing".


Expected results:

job.raw_stdout contains the actual raw STDOUT from the Tower job.


Additional info:

This behavior seems to be caused by the way the ansible tower gem being used by CFME executes the REST call to obtain stdout from the job.  The stdout for jobs is evidently not part of the data stored in the celery db, and instead resides solely on the executing server.  To account for this the Ansible Tower Rest API offers a format option to the request which forces the Tower server receiving the REST request to copy the stdout from the executing Tower server in order to provide it.

By changing the format argument from "txt" to "txt_download" the REST call to Tower consistently provides the correct stdout, however there does not seem to be an obvious way to alter the behavior of CFME to change the format option for the REST call to Ansible Tower.

Here is an illustration using CURL.

Assuming an Tower Job ID of 547.  (For reference, the job ID used internally by CFME and the one used by Tower are different - this example uses the latter).

Unreliable:


curl -f -k -H 'Content-Type: application/json' -XGET --user username:password https://ansible.acme.com/api/v1/jobs/547/stdout/?format=txt


Reliable:


curl -f -k -H 'Content-Type: application/json' -XGET --user username:password https://ansible.acme.com/api/v1/jobs/547/stdout/?format=txt_download
Comment 2 Dustin Scott 2017-07-03 09:23:14 EDT
FYI, the latest version of the ansible-tower-client rubygem offers this functionality:

https://github.com/ansible/ansible_tower_client_ruby/blob/master/lib/ansible_tower_client/base_models/job.rb


    def stdout(format = 'txt')
      out_url = related['stdout']
      return unless out_url
      api.get("#{out_url}?format=#{format}").body
    end


Would need to modify the format variable to utilize txt_download.
Comment 3 Bill Wei 2017-11-20 15:47:21 EST
With newer version (5.8 or newer) you can get txt_download format from rails console

job.raw_stdout('txt_download')

Note You need to log in before you can comment on or make changes to this bug.