Bug 1842291

Summary:	Ansible Default Template is running role dependency on the begin of every dependency role.
Product:	Red Hat Satellite	Reporter:	Rudnei Bertol Jr. <rbertolj>
Component:	Ansible	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Sam Bible <sbible>
Severity:	high	Docs Contact:
Priority:	high
Version:	6.6.0	CC:	aruzicka, dsynk, hakon.gislason, nalfassi, oezr, onerleka, pmendezh, pwaghmar
Target Milestone:	Unspecified	Keywords:	Triaged
Target Release:	Unused
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-01-23 16:29:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Rudnei Bertol Jr. 2020-05-31 15:55:51 UTC

Description of problem:

Ansible would only play dependent roles once. After upgrading, it plays dependent roles before every role, which causes it to play them multiple times when running roles on hosts.


Version-Release number of selected component (if applicable):

Satellite 6.7

How reproducible:

A complete reproducer will be provided on further comment.


Steps to Reproduce:
1.
2.
3.

Actual results:

When a host has roles and those roles have the same dependency, the ansible is running the dependency roles every time before the role, as the following example. 

Host 1 with ansible roles 'ansible_role_1','ansible_role_2','ansible_role_3'

Running ansible role ansible_role_dependency
Running ansible role ansible_role_1
Running ansible role ansible_role_dependency
Running ansible role ansible_role_2
Running ansible role ansible_role_dependency
Running ansible role ansible_role_3

This makes ansible terribly slow and repetitive.

Expected results:

The dependency role would be executed just once if it is a dependency for other roles.

Running ansible role ansible_role_dependency
Running ansible role ansible_role_1
Running ansible role ansible_role_2
Running ansible role ansible_role_3

Additional info:

Comment 1 Rudnei Bertol Jr. 2020-05-31 15:58:33 UTC

/// repro notes ///

Roles and dependencies.

~~~
[root@dell-r430-1 ansible]# ls -lht roles/
total 0
drwxr-xr-x. 10 root root 135 May 30 18:11 role2
drwxr-xr-x. 10 root root 135 May 30 18:11 role1
drwxr-xr-x. 10 root root 135 May 30 18:11 dependency_roles

[root@dell-r430-1 ansible]# grep depen roles/role1/meta/main.yml -B1

dependencies: 
  - role: dependency_roles
  # List your role dependencies here, one per line. Be sure to remove the '[]' above,
  # if you add dependencies to this list.

[root@dell-r430-1 ansible]# grep depen roles/role2/meta/main.yml -B1

dependencies:
  - role: dependency_roles
  # List your role dependencies here, one per line. Be sure to remove the '[]' above,
  # if you add dependencies to this list.

[root@dell-r430-1 ansible]# cat roles/role1/tasks/main.yml 
---
# tasks file for role1
- debug:
    msg: Running the role 1

[root@dell-r430-1 ansible]# cat roles/role2/tasks/main.yml 
---
# tasks file for role2
- debug:
    msg: Running the role 2

[root@dell-r430-1 ansible]# cat roles/dependency_roles/tasks/main.yml 
---
# tasks file for dependency_roles
- debug:
    msg: Running the dependency role
~~~

Roles assigned on the hosts.

~~~
[root@dell-r430-1 ansible]# hammer host ansible-roles list --name fake001.gsslab.rdu2.redhat.com
---|-------|--------------------
ID | NAME  | IMPORTED AT        
---|-------|--------------------
1  | role1 | 2020/05/30 22:17:37
---|-------|--------------------
[root@dell-r430-1 ansible]# hammer host ansible-roles list --name fake002.gsslab.rdu2.redhat.com
---|-------|--------------------
ID | NAME  | IMPORTED AT        
---|-------|--------------------
2  | role2 | 2020/05/30 22:17:37
---|-------|--------------------
[root@dell-r430-1 ansible]# hammer host ansible-roles list --name fake003.gsslab.rdu2.redhat.com
---|-------|--------------------
ID | NAME  | IMPORTED AT        
---|-------|--------------------
1  | role1 | 2020/05/30 22:17:37
2  | role2 | 2020/05/30 22:17:37
---|-------|--------------------
~~~

###### Target: fake001.gsslab.rdu2.redhat.com

Scroll to bottom
   1:
   2:PLAY [all] *********************************************************************
   3:
   4:TASK [Gathering Facts] *********************************************************
   5:422 Client Error: Unprocessable Entity
   6:422 Client Error: Unprocessable Entity
   7:ok: [fake001.gsslab.rdu2.redhat.com]
   8:422 Client Error: Unprocessable Entity
   9:
  10:TASK [Display all parameters known for the Foreman host] ***********************
  11:ok: [fake001.gsslab.rdu2.redhat.com] => {
  12:    "foreman": {
  13:        "content_view": null, 
  14:        "content_view_info": {}, 
  15:        "domainname": "gsslab.rdu2.redhat.com", 
  16:        "foreman_config_groups": [], 
  17:        "foreman_host_collections": [], 
  18:        "foreman_interfaces": [
  19:            {
  20:                "attached_to": null, 
  21:                "attrs": {}, 
  22:                "identifier": "", 
  23:                "ip": "127.0.0.1", 
  24:                "ip6": "", 
  25:                "link": true, 
  26:                "mac": "00:00:00:00:00:00", 
  27:                "managed": true, 
  28:                "name": "fake001.gsslab.rdu2.redhat.com", 
  29:                "primary": true, 
  30:                "provision": true, 
  31:                "subnet": null, 
  32:                "subnet6": null, 
  33:                "tag": null, 
  34:                "type": "Interface", 
  35:                "virtual": false
  36:            }
  37:        ], 
  38:        "foreman_subnets": [], 
  39:        "foreman_users": {
  40:            "admin": {
  41:                "description": null, 
  42:                "firstname": "Admin", 
  43:                "fullname": "Admin User", 
  44:                "lastname": "User", 
  45:                "mail": "root.redhat.com", 
  46:                "name": "admin", 
  47:                "ssh_authorized_keys": []
  48:            }
  49:        }, 
  50:        "kt_cv": null, 
  51:        "kt_env": null, 
  52:        "lifecycle_environment": null, 
  53:        "location": "Roadrunner", 
  54:        "location_title": "Roadrunner", 
  55:        "organization": "ACME", 
  56:        "organization_title": "ACME", 
  57:        "owner_email": "root.redhat.com", 
  58:        "owner_name": "Admin User", 
  59:        "puppetmaster": "", 
  60:        "root_pw": "$5$zoWTyKTUpxk4QDJf$c5wMiw3SEbYru1ZlZ2HSyJ13VrscopWe06zuJcMBA61", 
  61:        "ssh_authorized_keys": []
  62:    }
  63:}
  64:
  65:TASK [Apply roles] *************************************************************
  66:
  67:TASK [dependecy_roles : debug] *************************************************
  68:ok: [fake001.gsslab.rdu2.redhat.com] => {
  69:    "msg": "Running the dependecy role"
  70:}
  71:
  72:TASK [role1 : debug] ***********************************************************
  73:ok: [fake001.gsslab.rdu2.redhat.com] => {
  74:    "msg": "Running the role 1"
  75:}
  76:
  77:TASK [dependecy_roles : debug] *************************************************
  78:
  79:TASK [role2 : debug] ***********************************************************
  80:PLAY RECAP *********************************************************************
  81:fake001.gsslab.rdu2.redhat.com : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
  82:Exit status: 0


###### Target: fake002.gsslab.rdu2.redhat.com

Scroll to bottom
   1:
   2:PLAY [all] *********************************************************************
   3:
   4:TASK [Gathering Facts] *********************************************************
   5:422 Client Error: Unprocessable Entity
   6:ok: [fake002.gsslab.rdu2.redhat.com]
   7:422 Client Error: Unprocessable Entity
   8:422 Client Error: Unprocessable Entity
   9:
  10:TASK [Display all parameters known for the Foreman host] ***********************
  11:ok: [fake002.gsslab.rdu2.redhat.com] => {
  12:    "foreman": {
  13:        "content_view": null, 
  14:        "content_view_info": {}, 
  15:        "domainname": "gsslab.rdu2.redhat.com", 
  16:        "foreman_config_groups": [], 
  17:        "foreman_host_collections": [], 
  18:        "foreman_interfaces": [
  19:            {
  20:                "attached_to": null, 
  21:                "attrs": {}, 
  22:                "identifier": "", 
  23:                "ip": "127.0.0.1", 
  24:                "ip6": "", 
  25:                "link": true, 
  26:                "mac": "00:00:00:00:00:00", 
  27:                "managed": true, 
  28:                "name": "fake002.gsslab.rdu2.redhat.com", 
  29:                "primary": true, 
  30:                "provision": true, 
  31:                "subnet": null, 
  32:                "subnet6": null, 
  33:                "tag": null, 
  34:                "type": "Interface", 
  35:                "virtual": false
  36:            }
  37:        ], 
  38:        "foreman_subnets": [], 
  39:        "foreman_users": {
  40:            "admin": {
  41:                "description": null, 
  42:                "firstname": "Admin", 
  43:                "fullname": "Admin User", 
  44:                "lastname": "User", 
  45:                "mail": "root.redhat.com", 
  46:                "name": "admin", 
  47:                "ssh_authorized_keys": []
  48:            }
  49:        }, 
  50:        "kt_cv": null, 
  51:        "kt_env": null, 
  52:        "lifecycle_environment": null, 
  53:        "location": "Roadrunner", 
  54:        "location_title": "Roadrunner", 
  55:        "organization": "ACME", 
  56:        "organization_title": "ACME", 
  57:        "owner_email": "root.redhat.com", 
  58:        "owner_name": "Admin User", 
  59:        "puppetmaster": "", 
  60:        "root_pw": "$5$dlJUkvPMa2NbvYT4$9U.XPgimz9PfaJuAWvNK4k1YamN8GnlsfEftKhL6rn7", 
  61:        "ssh_authorized_keys": []
  62:    }
  63:}
  64:
  65:TASK [Apply roles] *************************************************************
  66:
  67:TASK [dependecy_roles : debug] *************************************************
  68:
  69:TASK [role1 : debug] ***********************************************************
  70:
  71:TASK [dependecy_roles : debug] *************************************************
  72:ok: [fake002.gsslab.rdu2.redhat.com] => {
  73:    "msg": "Running the dependecy role"
  74:}
  75:
  76:TASK [role2 : debug] ***********************************************************
  77:ok: [fake002.gsslab.rdu2.redhat.com] => {
  78:    "msg": "Running the role 2"
  79:}
  80:PLAY RECAP *********************************************************************
  81:fake002.gsslab.rdu2.redhat.com : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
  82:Exit status: 0

###### Target: fake003.gsslab.rdu2.redhat.com

Scroll to bottom
   1:
   2:PLAY [all] *********************************************************************
   3:
   4:TASK [Gathering Facts] *********************************************************
   5:ok: [fake003.gsslab.rdu2.redhat.com]
   6:422 Client Error: Unprocessable Entity
   7:422 Client Error: Unprocessable Entity
   8:422 Client Error: Unprocessable Entity
   9:
  10:TASK [Display all parameters known for the Foreman host] ***********************
  11:ok: [fake003.gsslab.rdu2.redhat.com] => {
  12:    "foreman": {
  13:        "content_view": null, 
  14:        "content_view_info": {}, 
  15:        "domainname": "gsslab.rdu2.redhat.com", 
  16:        "foreman_config_groups": [], 
  17:        "foreman_host_collections": [], 
  18:        "foreman_interfaces": [
  19:            {
  20:                "attached_to": null, 
  21:                "attrs": {}, 
  22:                "identifier": "", 
  23:                "ip": "127.0.0.1", 
  24:                "ip6": "", 
  25:                "link": true, 
  26:                "mac": "00:00:00:00:00:00", 
  27:                "managed": true, 
  28:                "name": "fake003.gsslab.rdu2.redhat.com", 
  29:                "primary": true, 
  30:                "provision": true, 
  31:                "subnet": null, 
  32:                "subnet6": null, 
  33:                "tag": null, 
  34:                "type": "Interface", 
  35:                "virtual": false
  36:            }
  37:        ], 
  38:        "foreman_subnets": [], 
  39:        "foreman_users": {
  40:            "admin": {
  41:                "description": null, 
  42:                "firstname": "Admin", 
  43:                "fullname": "Admin User", 
  44:                "lastname": "User", 
  45:                "mail": "root.redhat.com", 
  46:                "name": "admin", 
  47:                "ssh_authorized_keys": []
  48:            }
  49:        }, 
  50:        "kt_cv": null, 
  51:        "kt_env": null, 
  52:        "lifecycle_environment": null, 
  53:        "location": "Roadrunner", 
  54:        "location_title": "Roadrunner", 
  55:        "organization": "ACME", 
  56:        "organization_title": "ACME", 
  57:        "owner_email": "root.redhat.com", 
  58:        "owner_name": "Admin User", 
  59:        "puppetmaster": "", 
  60:        "root_pw": "$5$DYRTRJZEJZ4EU6st$mK71ISm7Dn2G6wCT1bgfDE6voIP87Zd2hus8InLI7gA", 
  61:        "ssh_authorized_keys": []
  62:    }
  63:}
  64:
  65:TASK [Apply roles] *************************************************************
  66:
  67:TASK [dependecy_roles : debug] *************************************************
  68:ok: [fake003.gsslab.rdu2.redhat.com] => {
  69:    "msg": "Running the dependecy role"
  70:}
  71:
  72:TASK [role1 : debug] ***********************************************************
  73:ok: [fake003.gsslab.rdu2.redhat.com] => {
  74:    "msg": "Running the role 1"
  75:}
  76:
  77:TASK [dependecy_roles : debug] *************************************************
  78:ok: [fake003.gsslab.rdu2.redhat.com] => {
  79:    "msg": "Running the dependecy role"
  80:}
  81:
  82:TASK [role2 : debug] ***********************************************************
  83:ok: [fake003.gsslab.rdu2.redhat.com] => {
  84:    "msg": "Running the role 2"
  85:}
  86:PLAY RECAP *********************************************************************
  87:fake003.gsslab.rdu2.redhat.com : ok=6    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
  88:Exit status: 0

regards
rbertol

Comment 2 Ondřej Ezr 2020-08-24 11:38:08 UTC

Created redmine issue https://projects.theforeman.org/issues/30691 from this bug

Comment 4 hakon.gislason 2020-11-19 10:35:44 UTC

Is this bug on any roadmap, or is there an ETA when work on this will begin?

Comment 7 Marek Hulan 2021-07-02 12:57:50 UTC

The question is, whether we could change such behavior for everyone, it could break the behavior for customers who rely on this already. Think about use case where you want to cleanup some cache before every role. It should happen every time, not just once.

I'm also not sure whether it should be Satellite that handles the deduplication. This is something that can be handled in the ansible roles (ansible code) today, customer can use Ansible handlers to register the need for a dependency at run it only once if the handler is registered. If they need to do it pior the actual role, the role can register a custom fact e.g. "dependency_xyz_ran" and check whether it's set to true in any run and skip the run if it's set to True already (already ran once).

I am dropping the Triaged keyword to be reconsidered.

Comment 8 Marek Hulan 2021-07-02 13:13:21 UTC

Actually after further consulting with engineer (namely oezr), this is the impact of how we dynamically include roles instead of importing, so that every host ends up in different role list. Therefore Ansible can't precalculate dependencies from _meta correctly. Reverting the Triaged keyword. If we change this somehow, we should consider to make the change optional because of the concern from comment 7

Comment 14 Brad Buckingham 2022-09-02 20:25:18 UTC

Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team.  Thank you.

Comment 15 Brad Buckingham 2022-09-02 20:31:56 UTC

Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team.  Thank you.