Description of problem: ==================== While performing upgrade from RHCS 5.3z1 --> IBM 5.3z1 , cephadm-preflight playbook is failing to upgrade the cephadm package on other nodes --------- [cephuser@ceph-dsms-0l2782-node1-installer cephadm-ansible]$ sudo ansible-playbook -i hosts cephadm-preflight.yml --extra-vars "ceph_origin=custom upgrade_ceph_packages=true custom_repo_url=http://ceph-dsms-0l2782-node1-installer/repo/IBM-CEPH-5.3-20230228.ci.0" [DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing names to new standard, use callbacks_enabled instead. This feature will be removed from ansible-core in version 2.15. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. [WARNING]: Skipping callback plugin 'profile_tasks', unable to load PLAY [localhost] ************************************************************************************************************************************************************ TASK [fail if insecure_registry is undefined] ******************************************************************************************************************************* skipping: [localhost] PLAY [localhost] ************************************************************************************************************************************************************ TASK [fail when ceph_origin is custom with no repository defined] *********************************************************************************************************** skipping: [localhost] TASK [fail if baseurl is not defined for ceph_custom_repositories] ********************************************************************************************************** skipping: [localhost] PLAY [all] ****************************************************************************************************************************************************************** TASK [Gathering Facts] ****************************************************************************************************************************************************** ok: [ceph-dsms-0l2782-node4] ok: [ceph-dsms-0l2782-node3] ok: [ceph-dsms-0l2782-node2] ok: [ceph-dsms-0l2782-node1-installer] TASK [enable red hat storage tools repository] ****************************************************************************************************************************** skipping: [ceph-dsms-0l2782-node1-installer] skipping: [ceph-dsms-0l2782-node3] skipping: [ceph-dsms-0l2782-node2] skipping: [ceph-dsms-0l2782-node4] TASK [set_fact _ceph_repo] ************************************************************************************************************************************************** skipping: [ceph-dsms-0l2782-node1-installer] skipping: [ceph-dsms-0l2782-node3] skipping: [ceph-dsms-0l2782-node2] skipping: [ceph-dsms-0l2782-node4] TASK [configure ceph repository key] **************************************************************************************************************************************** skipping: [ceph-dsms-0l2782-node1-installer] skipping: [ceph-dsms-0l2782-node3] skipping: [ceph-dsms-0l2782-node2] skipping: [ceph-dsms-0l2782-node4] TASK [configure ceph stable repository] ************************************************************************************************************************************* skipping: [ceph-dsms-0l2782-node1-installer] skipping: [ceph-dsms-0l2782-node3] skipping: [ceph-dsms-0l2782-node2] skipping: [ceph-dsms-0l2782-node4] TASK [fetch ceph red hat development repository] **************************************************************************************************************************** skipping: [ceph-dsms-0l2782-node1-installer] skipping: [ceph-dsms-0l2782-node3] skipping: [ceph-dsms-0l2782-node2] skipping: [ceph-dsms-0l2782-node4] TASK [configure ceph red hat development repository] ************************************************************************************************************************ skipping: [ceph-dsms-0l2782-node1-installer] skipping: [ceph-dsms-0l2782-node3] skipping: [ceph-dsms-0l2782-node2] skipping: [ceph-dsms-0l2782-node4] TASK [set_fact ceph_custom_repositories] ************************************************************************************************************************************ ok: [ceph-dsms-0l2782-node1-installer] ok: [ceph-dsms-0l2782-node3] ok: [ceph-dsms-0l2782-node2] ok: [ceph-dsms-0l2782-node4] TASK [setup custom repositories] ******************************************************************************************************************************************** ok: [ceph-dsms-0l2782-node1-installer] => (item={'name': 'ceph_custom', 'description': 'Ceph custom repo', 'gpgcheck': 'no', 'state': 'present', 'baseurl': 'http://ceph-dsms-0l2782-node1-installer/repo/IBM-CEPH-5.3-20230228.ci.0', 'enabled': '1', 'file': 'ceph_custom', 'priority': '2'}) ok: [ceph-dsms-0l2782-node2] => (item={'name': 'ceph_custom', 'description': 'Ceph custom repo', 'gpgcheck': 'no', 'state': 'present', 'baseurl': 'http://ceph-dsms-0l2782-node1-installer/repo/IBM-CEPH-5.3-20230228.ci.0', 'enabled': '1', 'file': 'ceph_custom', 'priority': '2'}) ok: [ceph-dsms-0l2782-node4] => (item={'name': 'ceph_custom', 'description': 'Ceph custom repo', 'gpgcheck': 'no', 'state': 'present', 'baseurl': 'http://ceph-dsms-0l2782-node1-installer/repo/IBM-CEPH-5.3-20230228.ci.0', 'enabled': '1', 'file': 'ceph_custom', 'priority': '2'}) ok: [ceph-dsms-0l2782-node3] => (item={'name': 'ceph_custom', 'description': 'Ceph custom repo', 'gpgcheck': 'no', 'state': 'present', 'baseurl': 'http://ceph-dsms-0l2782-node1-installer/repo/IBM-CEPH-5.3-20230228.ci.0', 'enabled': '1', 'file': 'ceph_custom', 'priority': '2'}) TASK [install epel-release] ************************************************************************************************************************************************* skipping: [ceph-dsms-0l2782-node3] skipping: [ceph-dsms-0l2782-node1-installer] skipping: [ceph-dsms-0l2782-node2] skipping: [ceph-dsms-0l2782-node4] TASK [install ceph-common on rhel] ****************************************************************************************************************************************** ok: [ceph-dsms-0l2782-node3] ok: [ceph-dsms-0l2782-node4] ok: [ceph-dsms-0l2782-node2] ok: [ceph-dsms-0l2782-node1-installer] TASK [install prerequisites packages] *************************************************************************************************************************************** ok: [ceph-dsms-0l2782-node1-installer] FAILED - RETRYING: [ceph-dsms-0l2782-node3]: install prerequisites packages (3 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node4]: install prerequisites packages (3 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node2]: install prerequisites packages (3 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node3]: install prerequisites packages (2 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node4]: install prerequisites packages (2 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node2]: install prerequisites packages (2 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node3]: install prerequisites packages (1 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node4]: install prerequisites packages (1 retries left). FAILED - RETRYING: [ceph-dsms-0l2782-node2]: install prerequisites packages (1 retries left). fatal: [ceph-dsms-0l2782-node3]: FAILED! => changed=false attempts: 3 failures: - - The IBM license agreement has not been accepted. Exiting. - Please install ibm-storage-ceph-license package first. - 'error: %pretrans(cephadm-2:16.2.10-140.el8cp.noarch) scriptlet failed, exit status 1' msg: Failed to install some of the specified packages rc: 1 results: [] fatal: [ceph-dsms-0l2782-node4]: FAILED! => changed=false attempts: 3 failures: - - The IBM license agreement has not been accepted. Exiting. - Please install ibm-storage-ceph-license package first. - 'error: %pretrans(cephadm-2:16.2.10-140.el8cp.noarch) scriptlet failed, exit status 1' msg: Failed to install some of the specified packages rc: 1 results: [] fatal: [ceph-dsms-0l2782-node2]: FAILED! => changed=false attempts: 3 failures: - - The IBM license agreement has not been accepted. Exiting. - Please install ibm-storage-ceph-license package first. - 'error: %pretrans(cephadm-2:16.2.10-140.el8cp.noarch) scriptlet failed, exit status 1' msg: Failed to install some of the specified packages rc: 1 results: [] TASK [ensure chronyd is running] ******************************************************************************************************************************************** ok: [ceph-dsms-0l2782-node1-installer] PLAY [localhost] ************************************************************************************************************************************************************ TASK [fail if insecure_registry is undefined] ******************************************************************************************************************************* skipping: [localhost] PLAY [all] ****************************************************************************************************************************************************************** TASK [fail if insecure_registry is undefined] ******************************************************************************************************************************* skipping: [ceph-dsms-0l2782-node1-installer] TASK [add registry as insecure registry in registries.conf] ***************************************************************************************************************** skipping: [ceph-dsms-0l2782-node1-installer] PLAY RECAP ****************************************************************************************************************************************************************** ceph-dsms-0l2782-node1-installer : ok=6 changed=0 unreachable=0 failed=0 skipped=9 rescued=0 ignored=0 ceph-dsms-0l2782-node2 : ok=4 changed=0 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0 ceph-dsms-0l2782-node3 : ok=4 changed=0 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0 ceph-dsms-0l2782-node4 : ok=4 changed=0 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0 localhost : ok=0 changed=0 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0 -------------- All packages were upgraded except cephadm package -------------- # rpm -qa | grep ceph libcephfs2-16.2.10-140.el8cp.x86_64 python3-ceph-common-16.2.10-140.el8cp.x86_64 cephadm-16.2.10-138.el8cp.noarch python3-cephfs-16.2.10-140.el8cp.x86_64 ceph-common-16.2.10-140.el8cp.x86_64 python3-ceph-argparse-16.2.10-140.el8cp.x86_64 ibm-storage-ceph-license-5.3-1.el8cp.noarch Version-Release number of selected component (if applicable): $ rpm -qa | grep cephadm cephadm-ansible-1.14.0-1.el8cp.noarch cephadm-16.2.10-140.el8cp.noarch How reproducible: Steps to Reproduce: ===================== 1.Install RHCS 5.3z1 cluster 2.Perform upgrade to IBM 5.3z1 a) Upgrade the cephadm package on installer node ==> Successfully upgraded b) Run preflight playbook to upgrade packages on other nodes Actual results: ============= preflight playbook failed complaining to accept the Licence first Expected results: =========== Playbook should pass Additional info:
Hi, I tried workaround of installing licencing package manually on all nodes and then run preflight playbook.In that case the playbook is passing.