Bug 1734904 - Embedded Ansible does not use the roles/requirements.yml file to locate a Galaxy role
Summary: Embedded Ansible does not use the roles/requirements.yml file to locate a Gal...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance
Version: 5.11.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.11.0
Assignee: Nick Carboni
QA Contact: Satyajit Bulage
Red Hat CloudForms Documentation
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-31 16:29 UTC by Peter McGowan
Modified: 2019-12-13 14:54 UTC (History)
9 users (show)

Fixed In Version: 5.11.0.18
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-13 14:54:40 UTC
Category: Bug
Cloudforms Team: Ansible
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Peter McGowan 2019-07-31 16:29:18 UTC
Description of problem:
If an embedded Ansible playbook references a Galaxy role, it should be retrieved and run if mentioned in the roles/requirements.yml file in the repository. For example a playbook with the following yaml:

  roles:
  - syncrou.manageiq-automate

should run successfully if there is a roles/requirements.yml file in the repository containing:

---
# from galaxy
- src: syncrou.manageiq-automate

Version-Release number of selected component (if applicable):

How reproducible:
5.11.0.17

Steps to Reproduce:
1. Create a playbook that references a Galaxy role
2. Create a roles/requirements.yml file in the root of the repository that references the role name
3. Run the playbook using embedded Ansible

Actual results:
The following is logged to evm.log:

[----] W, [2019-07-31T17:14:06.995894 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at 'ERROR! the role 'syncrou.manageiq-automate' was not found in /tmp/ansible-playbook-repo20190731-8931-x3qhk7/roles:/root/.ansible/roles:/usr/share/ans
ible/roles:/etc/ansible/roles:/tmp/ansible-playbook-repo20190731-8931-x3qhk7
'
[----] W, [2019-07-31T17:14:06.996048 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at ''
[----] W, [2019-07-31T17:14:06.996104 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at 'The error appears to be in '/tmp/ansible-playbook-repo20190731-8931-x3qhk7/manageiq-automate_role_bug_example.yml': line 8, column 5, but may
'
[----] W, [2019-07-31T17:14:06.996155 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at 'be elsewhere in the file depending on the exact syntax problem.
'
[----] W, [2019-07-31T17:14:06.996225 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at ''
[----] W, [2019-07-31T17:14:06.996284 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at 'The offending line appears to be:
'
[----] W, [2019-07-31T17:14:06.996325 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at ''
[----] W, [2019-07-31T17:14:06.996365 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at 'roles:
'
[----] W, [2019-07-31T17:14:06.996412 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at '- syncrou.manageiq-automate
'
[----] W, [2019-07-31T17:14:06.996462 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(Ansible::Runner::Response#parse_stdout) Couldn't parse JSON from: 765: unexpected token at '^ here
'
[----] W, [2019-07-31T17:14:07.036836 #8931:2ac23e5fe5bc]  WARN -- : Q-task_id([a2edf7a3-1c41-4f72-a0b7-ebe40d385259]) MIQ(ManageIQ::Providers::AnsiblePlaybookWorkflow#poll_runner) Playbook failed:

Expected results:
The Galaxy role should be downloaded and run

Additional info:
This is a regression from CFME 5.10, where the roles/requirements.yml file is used by AWX to locate a role (https://www.ansible.com/blog/using-ansible-and-ansible-tower-with-shared-roles)

Comment 2 Nick Carboni 2019-07-31 17:19:18 UTC
This should be fixed by https://github.com/ManageIQ/manageiq/pull/19079

Comment 3 CFME Bot 2019-08-01 19:55:48 UTC
New commit detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/6bfcc74e23e42a11bd6b263e24c6b161b9309dd2
commit 6bfcc74e23e42a11bd6b263e24c6b161b9309dd2
Author:     Nick Carboni <ncarboni>
AuthorDate: Wed Jul 31 16:24:05 2019 -0400
Commit:     Nick Carboni <ncarboni>
CommitDate: Wed Jul 31 16:24:05 2019 -0400

    Move logic to fetch galaxy roles to Ansible::Runner

    This is needed because the roles directory location is located
    relative to the playbook, not always the root of the repo.

    This will allow us to handle a case where a repo is configured with
    the following directory structure:

    azure_servers/
      roles/
        azure_stuffs/
        requirements.yml
      azure_playbook.yml
    aws_servers/
      roles/
        s3_stuffs/
        ec2_config/
      aws_playbook.yml
      s3_playbook.yml
    localhost/
      requirements.yml
      localhost_playbook.yml

    Where previously we would not have installed any roles because we
    wouldn't have found a /roles directory at the top-level of the
    repo directory.

    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1734904

 app/models/manageiq/providers/embedded_ansible/automation_manager/configuration_script_source.rb | 1 -
 lib/ansible/runner.rb | 8 +
 spec/lib/ansible/runner_spec.rb | 14 +
 3 files changed, 22 insertions(+), 1 deletion(-)

Comment 4 CFME Bot 2019-08-01 22:40:47 UTC
New commit detected on ManageIQ/manageiq/ivanchuk:

https://github.com/ManageIQ/manageiq/commit/1990eb4358ac3c20ceccf45166da17966c6b4c9f
commit 1990eb4358ac3c20ceccf45166da17966c6b4c9f
Author:     Jason Frey <jfrey>
AuthorDate: Thu Aug  1 15:51:48 2019 -0400
Commit:     Jason Frey <jfrey>
CommitDate: Thu Aug  1 15:51:48 2019 -0400

    Merge pull request #19079 from carbonin/pull_roles_from_ansible_galaxy

    Pull roles from ansible galaxy before running a playbook

    (cherry picked from commit e783031e055ccfffda09dc1b1f2c51a557710b95)

    https://bugzilla.redhat.com/show_bug.cgi?id=1734904

 lib/ansible/content.rb | 22 +-
 lib/ansible/runner.rb | 14 +
 spec/lib/ansible/content_spec.rb | 73 +
 spec/lib/ansible/runner_spec.rb | 24 +-
 4 files changed, 124 insertions(+), 9 deletions(-)

Comment 6 Satyajit Bulage 2019-08-12 09:31:40 UTC
Playbook used for verification of this BZ --> https://github.com/sbulage/ansible_playbooks/blob/master/ansible_galaxy_role_users.yaml
Able to see the roles is fetched from Ansible Galaxy without any issue.

Verified Version: 5.11.0.18.20190806180636_1dd6378


Note You need to log in before you can comment on or make changes to this bug.