Bug 2148827

Summary: [Cephadm] [RHCS 6.0] cephadm-preflight.yml playbook is failing for disconnected install having custom repo on RHEL 9
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Manisha Saini <msaini>
Component: CephadmAssignee: Guillaume Abrioux <gabrioux>
Status: CLOSED ERRATA QA Contact: Manisha Saini <msaini>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.0CC: akraj, cephqe-warriors, ekristov, gabrioux, kdreyer, rmandyam, vdas, vereddy
Target Milestone: ---   
Target Release: 6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: cephadm-ansible-2.11.0-1.el9cp Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-03-20 18:59:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Manisha Saini 2022-11-28 04:52:31 UTC 
Description of problem:
=======================
cephadm-preflight.yml is failing while upgrading the ceph with disconnected installaton from RHCS 5.3+ RHEL 8 to RHCS 6.0+ RHEL 9


[cephuser@ceph-msaini-x723pp-node1-installer cephadm-ansible]$ ansible-playbook -i hosts cephadm-preflight.yml --extra-vars "ceph_origin=custom" -e "custom_repo_url=http://ceph-msaini-x723pp-node8/repo/ceph6-rhel9"
[DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing names to new standard, use callbacks_enabled instead. This feature will be removed from ansible-core in version 2.15. Deprecation warnings can be disabled by setting
 deprecation_warnings=False in ansible.cfg.
[WARNING]: Skipping callback plugin 'profile_tasks', unable to load

PLAY [localhost] *****************************************************************************************************************************************************************************************************************************

TASK [fail if insecure_registry is undefined] ************************************************************************************************************************************************************************************************
skipping: [localhost]

PLAY [localhost] *****************************************************************************************************************************************************************************************************************************

TASK [fail when ceph_origin is custom with no repository defined] ****************************************************************************************************************************************************************************
skipping: [localhost]

TASK [fail if baseurl is not defined for ceph_custom_repositories] ***************************************************************************************************************************************************************************
skipping: [localhost]

PLAY [all] ***********************************************************************************************************************************************************************************************************************************

TASK [enable red hat storage tools repository] ***********************************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer]
skipping: [ceph-msaini-X723PP-node2]
skipping: [ceph-msaini-X723PP-node3]
skipping: [ceph-msaini-X723PP-node4]
skipping: [ceph-msaini-X723PP-node5]
skipping: [ceph-msaini-X723PP-node6]
skipping: [ceph-msaini-X723PP-node7]

TASK [disable older rhceph repositories if any] **********************************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer] => (item=rhceph-4-tools-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node1-installer] => (item=rhceph-4-mon-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node1-installer] => (item=rhceph-4-osd-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node2] => (item=rhceph-4-tools-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node2] => (item=rhceph-4-mon-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node2] => (item=rhceph-4-osd-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node3] => (item=rhceph-4-tools-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node3] => (item=rhceph-4-mon-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node3] => (item=rhceph-4-osd-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node4] => (item=rhceph-4-tools-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node4] => (item=rhceph-4-mon-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node4] => (item=rhceph-4-osd-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node5] => (item=rhceph-4-tools-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node5] => (item=rhceph-4-mon-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node5] => (item=rhceph-4-osd-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node6] => (item=rhceph-4-tools-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node6] => (item=rhceph-4-mon-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node6] => (item=rhceph-4-osd-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node7] => (item=rhceph-4-tools-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node7] => (item=rhceph-4-mon-for-rhel-8-x86_64-rpms) 
skipping: [ceph-msaini-X723PP-node7] => (item=rhceph-4-osd-for-rhel-8-x86_64-rpms) 

TASK [configure red hat ceph community repository stable key] ********************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer]
skipping: [ceph-msaini-X723PP-node2]
skipping: [ceph-msaini-X723PP-node3]
skipping: [ceph-msaini-X723PP-node4]
skipping: [ceph-msaini-X723PP-node5]
skipping: [ceph-msaini-X723PP-node6]
skipping: [ceph-msaini-X723PP-node7]

TASK [configure red hat ceph stable community repository] ************************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer]
skipping: [ceph-msaini-X723PP-node2]
skipping: [ceph-msaini-X723PP-node3]
skipping: [ceph-msaini-X723PP-node4]
skipping: [ceph-msaini-X723PP-node5]
skipping: [ceph-msaini-X723PP-node6]
skipping: [ceph-msaini-X723PP-node7]

TASK [configure red hat ceph stable noarch community repository] *****************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer]
skipping: [ceph-msaini-X723PP-node2]
skipping: [ceph-msaini-X723PP-node3]
skipping: [ceph-msaini-X723PP-node4]
skipping: [ceph-msaini-X723PP-node5]
skipping: [ceph-msaini-X723PP-node6]
skipping: [ceph-msaini-X723PP-node7]

TASK [fetch ceph red hat development repository] *********************************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer]
skipping: [ceph-msaini-X723PP-node2]
skipping: [ceph-msaini-X723PP-node3]
skipping: [ceph-msaini-X723PP-node4]
skipping: [ceph-msaini-X723PP-node5]
skipping: [ceph-msaini-X723PP-node6]
skipping: [ceph-msaini-X723PP-node7]

TASK [configure ceph red hat development repository] *****************************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer]
skipping: [ceph-msaini-X723PP-node2]
skipping: [ceph-msaini-X723PP-node3]
skipping: [ceph-msaini-X723PP-node4]
skipping: [ceph-msaini-X723PP-node5]
skipping: [ceph-msaini-X723PP-node6]
skipping: [ceph-msaini-X723PP-node7]

TASK [remove ceph_stable repositories] *******************************************************************************************************************************************************************************************************
skipping: [ceph-msaini-X723PP-node1-installer] => (item=ceph_stable) 
skipping: [ceph-msaini-X723PP-node1-installer] => (item=ceph_stable_noarch) 
skipping: [ceph-msaini-X723PP-node2] => (item=ceph_stable) 
skipping: [ceph-msaini-X723PP-node2] => (item=ceph_stable_noarch) 
skipping: [ceph-msaini-X723PP-node3] => (item=ceph_stable) 
skipping: [ceph-msaini-X723PP-node3] => (item=ceph_stable_noarch) 
skipping: [ceph-msaini-X723PP-node4] => (item=ceph_stable) 
skipping: [ceph-msaini-X723PP-node4] => (item=ceph_stable_noarch) 
skipping: [ceph-msaini-X723PP-node5] => (item=ceph_stable) 
skipping: [ceph-msaini-X723PP-node5] => (item=ceph_stable_noarch) 
skipping: [ceph-msaini-X723PP-node6] => (item=ceph_stable) 
skipping: [ceph-msaini-X723PP-node6] => (item=ceph_stable_noarch) 
skipping: [ceph-msaini-X723PP-node7] => (item=ceph_stable) 
skipping: [ceph-msaini-X723PP-node7] => (item=ceph_stable_noarch) 

TASK [set_fact ceph_custom_repositories] *****************************************************************************************************************************************************************************************************
ok: [ceph-msaini-X723PP-node1-installer]
ok: [ceph-msaini-X723PP-node2]
ok: [ceph-msaini-X723PP-node3]
ok: [ceph-msaini-X723PP-node4]
ok: [ceph-msaini-X723PP-node6]
ok: [ceph-msaini-X723PP-node5]
ok: [ceph-msaini-X723PP-node7]

TASK [setup custom repositories] *************************************************************************************************************************************************************************************************************
fatal: [ceph-msaini-X723PP-node1-installer]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node2]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node3]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node4]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node5]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node6]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node7]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
ceph-msaini-X723PP-node1-installer : ok=1    changed=0    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0   
ceph-msaini-X723PP-node2   : ok=1    changed=0    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0   
ceph-msaini-X723PP-node3   : ok=1    changed=0    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0   
ceph-msaini-X723PP-node4   : ok=1    changed=0    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0   
ceph-msaini-X723PP-node5   : ok=1    changed=0    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0   
ceph-msaini-X723PP-node6   : ok=1    changed=0    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0   
ceph-msaini-X723PP-node7   : ok=1    changed=0    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0   
localhost                  : ok=0    changed=0    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0   


Version-Release number of selected component (if applicable):
==================
[cephuser@ceph-msaini-x723pp-node1-installer ~]$ rpm -qa | grep cephadm
cephadm-17.2.5-14.el9cp.noarch
cephadm-ansible-2.10.0-1.el9cp.noarch


How reproducible:
================
1/1


Steps to Reproduce:
1.Deploy CEPH 5.3+ RHEL 8 with disconnected installation
2. Perform host OS upgrade from RHEL 8 to RHEL 9
3. Perform ceph upgrade

--> Upgrade cephadm and ceph-ansible package on installer node   

-->  ansible-playbook -vvv -i hosts cephadm-preflight.yml --extra-vars "ceph_origin=custom upgrade_ceph_packages=true -e custom_repo_url=http://ceph-msaini-x723pp-node8/repo/ceph6-rhel9"


Actual results:
===============
cephadm-preflight.yml is failing for custom repo


Expected results:
================
cephadm-preflight.yml playbook should not fail


Additional info:
============
Seems to be a typo error "defualt" in playbook. As a result playbook fail. 

TASK [setup custom repositories] *************************************************************************************************************************************************************************************************************
fatal: [ceph-msaini-X723PP-node1-installer]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node2]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node3]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node4]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node5]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node6]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
fatal: [ceph-msaini-X723PP-node7]: FAILED! => 
  msg: 'template error while templating string: No filter named ''defualt''.. String: {{ item.enabled | defualt(omit) }}'
Comment 17 errata-xmlrpc 2023-03-20 18:59:40 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 6.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:1360