Bug 2027052

Description of problem:

The satellite inventory collection plugin redhat.satellite.foreman is unable to fetch all system facts by default and Mainly the custom facts uploaded by ansible for the system. 


Version-Release number of selected component (if applicable):

redhat.satellite collection 2.0.1 - 2.2.0 

How reproducible:
Always

Steps to Reproduce:

1) Assume:
  Satellite: satellite.example.com
  Client host: rhel8.example.com


2) SSH to client system rhel8.example.com and do the following.

# mkdir -p /etc/ansible/facts.d

# vi /etc/ansible/facts.d/bdem.fact 

# cat /etc/ansible/facts.d/bdem.fact 
[bdem_custom_facts]
bdem_environment = DEV
bdem_cust_fact = Testje2


3) SSH to Satellite server and then:

A) Setup a local ansible configuration with callback enabled and redhat.collection installed from galaxy

# cat inventory
[rhel8]
rhel8.example.com

# cat ansible.cfg
[defaults]
inventory = ./inventory
remote_tmp = /tmp/ansible-$USER
ask_pass = False
host_key_checking = False
remote_user = root
timeout = 30
callback_whitelist = foreman
local_tmp = /tmp
host_key_checking = False
stdout_callback = yaml
roles_path = /etc/ansible/roles:/usr/share/ansible/roles
collections_paths = ./collections

[callback_foreman]
url = https://satellite.example.com
ssl_cert = /etc/foreman-proxy/foreman_ssl_cert.pem
ssl_key = /etc/foreman-proxy/foreman_ssl_key.pem
verify_certs = /etc/foreman-proxy/foreman_ssl_ca.pem

[ssh_connection]
ssh_args = -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s

[privilege_escalation]
become = True
become_method = sudo
become_user = root
become_ask_pass = False


[galaxy]

server_list = automation_hub, release_galaxy

[galaxy_server.automation_hub]
url=https://cloud.redhat.com/api/automation-hub/
auth_url=xxxxxxxx

[galaxy_server.release_galaxy]
url=https://galaxy.ansible.com/
token=yyyyyy

# ansible-galaxy collection install  redhat.satellite


B) Create a play to grab those custom ansible facts from rhel8 client system and upload them to satellite.

# cat foreman_update_facts.yml
- name: gather ansible facts and push them to foreman
  hosts: rhel8
  gather_facts: true

  tasks:
    - name: display facts
      debug: 
        var: ansible_facts.ansible_local.bdem



# ansible-playbook foreman_update_facts.yml 

PLAY [gather ansible facts and push them to foreman] ********************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************
ok: [rhel8.example.com]

TASK [display facts] ****************************************************************************************************************************************************
ok: [rhel8.example.com] => 
  ansible_facts.ansible_local.bdem:
    bdem_custom_facts:
      bdem_cust_fact: Testje2
      bdem_environment: DEV

PLAY RECAP **************************************************************************************************************************************************************
rhel8.example.com : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   



# ansible rhel8 -m setup | grep bdem 
            "bdem": {
                "bdem_custom_facts": {
                    "bdem_cust_fact": "Testje2", 
                    "bdem_environment": "DEV"



# hammer host facts --name rhel8.example.com | grep bdem
ansible_local::bdem                                             |                                                                                 
ansible_local::bdem::bdem_custom_facts                          |                                                                                 
ansible_local::bdem::bdem_custom_facts::bdem_environment        | DEV                                                                             
ansible_local::bdem::bdem_custom_facts::bdem_cust_fact          | Testje2                                                                         


C) Setup the inventory plugin to collect the inventory of Satellite i.e. in this case, just the details from that rhel 8 system.

# cat my.foreman.yml
plugin: redhat.satellite.foreman

url: https://satellite.example.com
user: admin
password: RedHat1!
validate_certs: no
host_filters: 'name ~ rhel8'
want_facts: yes
want_params: yes
want_host_group: no
want_hostcollections: no
want_location: no
want_organization: no
#batch_size: 10000
#use_reports_api: true

keyed_groups:
  - prefix: "bdm"
    separator: "_"
    key: foreman_content_facet_attributes.content_view_name
  - prefix: bdm_cust
    separator: "_"
    key: ansible_local.bdem.bdem_custom_facts.bdem_cust_fact 
                                                                   


D) Execute the ansible-inventory command line to collect the data.

# ansible-inventory -i my.foreman.yml --host rhel8.example.com  | grep -i bdem


Actual results:

The custom facts are not properly collected. 

# ansible-inventory -i my.foreman.yml --host rhel8.example.com  | grep -i bdem

        "ansible_local::bdem": null, 
        "ansible_local::bdem::bdem_custom_facts": null,



Expected results:

# ansible-inventory -i my.foreman.yml --host rhel8.example.com  | grep -i bdem
        "ansible_local::bdem": null, 
        "ansible_local::bdem::bdem_custom_facts": null, 
        "ansible_local::bdem::bdem_custom_facts::bdem_cust_fact": "Testje2", 
        "ansible_local::bdem::bdem_custom_facts::bdem_environment": "DEV",


Additional info:

** If inside my.foreman.yml, we use "use_reports_api: true" , then it will show the expected results but not with Default API.

** The batch_size parameter is set to 250 by default. I earlier thought, It's being used to limit the number of hosts per batch being retrieved from foreman but it seems like, that's not the only case. It's also limiting the amount of facts per host to 250 it seems. 

So if we use some bigger value for it i.e. "batch_size: 10000", then also, the inventory plugin will be able to fetch > 250 facts from the hosts in foreman\satellite and you should be able to see the custom facts as well.


The patch https://github.com/theforeman/foreman-ansible-modules/pull/1321 fixes the issue with default API and batch_size. So no need to use any of the two workarounds if we use the patch from https://github.com/theforeman/foreman-ansible-modules/pull/1321 .  I have tested and confirmed the patch works great with redhat.satellite collection 2.0.1 - 2.2.0 .