Red Hat Bugzilla – Bug 1464518
raw_stdout member of Ansible Tower job object is set to "stdout capture is missing" when using HA Tower
Last modified: 2017-11-20 15:47:21 EST
Description of problem:
When using Ansible Tower in an HA configuration, the raw_stdout member of the job object is set to "stdout capture is missing" if job refresh was performed against any tower server other than the one where the job was run.
Version-Release number of selected component (if applicable):
Any time an Ansible Tower Job is executed from Automate against an Ansible Tower management provider in an HA mode, there is a proportional liklihood that the output of the job's raw_stdout member will be erroneously set to "stdout capture is missing".
If the job refresh request was run
Steps to Reproduce:
1. Launch an Ansible Tower Job Template from an Automate workflow, record the ansible job ID.
2. From a method within Automate OR the CFME rails console, reference the job object with the job ID recorded in step 1.
JOB_CLASS = 'ManageIQ_Providers_AnsibleTower_ConfigurationManager_Job'.freeze
job = $evm.vmdb(JOB_CLASS).find(job_id)
3. Attempt to reference the job object's raw_stdout method
If the most recent refresh was performed against a non-executing Tower server (i.e. any server in the HA configuration other than the one that executed the job) the output will be set to "stdout capture is missing".
job.raw_stdout contains the actual raw STDOUT from the Tower job.
This behavior seems to be caused by the way the ansible tower gem being used by CFME executes the REST call to obtain stdout from the job. The stdout for jobs is evidently not part of the data stored in the celery db, and instead resides solely on the executing server. To account for this the Ansible Tower Rest API offers a format option to the request which forces the Tower server receiving the REST request to copy the stdout from the executing Tower server in order to provide it.
By changing the format argument from "txt" to "txt_download" the REST call to Tower consistently provides the correct stdout, however there does not seem to be an obvious way to alter the behavior of CFME to change the format option for the REST call to Ansible Tower.
Here is an illustration using CURL.
Assuming an Tower Job ID of 547. (For reference, the job ID used internally by CFME and the one used by Tower are different - this example uses the latter).
curl -f -k -H 'Content-Type: application/json' -XGET --user username:password https://ansible.acme.com/api/v1/jobs/547/stdout/?format=txt
curl -f -k -H 'Content-Type: application/json' -XGET --user username:password https://ansible.acme.com/api/v1/jobs/547/stdout/?format=txt_download
FYI, the latest version of the ansible-tower-client rubygem offers this functionality:
def stdout(format = 'txt')
out_url = related['stdout']
return unless out_url
Would need to modify the format variable to utilize txt_download.
With newer version (5.8 or newer) you can get txt_download format from rails console