Bug 2093414 - Upgrade [OSP16.2 -> OSP17.1] ceph upgrade failing due to "No file was found when using first_found"
Summary: Upgrade [OSP16.2 -> OSP17.1] ceph upgrade failing due to "No file was found w...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: tripleo-ansible
Version: 17.0 (Wallaby)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 17.1
Assignee: Francesco Pantano
QA Contact: Juan Badia Payno
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-06-03 16:06 UTC by Juan Badia Payno
Modified: 2023-08-16 01:11 UTC (History)
7 users (show)

Fixed In Version: tripleo-ansible-3.3.1-0.20220615001845.000acea.el8ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-08-16 01:11:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 844864 0 None master: MERGED tripleo-ansible: Remove gathering facts (I5c0cf36178511d286e89c3eee907474f68d14fad) 2022-06-17 14:05:27 UTC
OpenStack gerrit 844937 0 None stable/wallaby: MERGED tripleo-ansible: Remove gathering facts (I5c0cf36178511d286e89c3eee907474f68d14fad) 2022-06-17 14:05:32 UTC
Red Hat Issue Tracker OSP-15532 0 None None None 2022-06-03 16:11:40 UTC
Red Hat Product Errata RHEA-2023:4577 0 None None None 2023-08-16 01:11:53 UTC

Description Juan Badia Payno 2022-06-03 16:06:36 UTC
After upgrading the undercloud from osp16.2 to osp17.0
Then the heat stack was updated.
 "openstack overcloud upgrade prepare" command
The the ceph upgrade from ceph4 to ceph5 was executed.
 "openstack overcloud external-upgrade --stack qe-Cloud-0 --tags ceph"

The following error was shown:

2022-05-28 11:47:08.729801 | 525400cf-ccf8-4dfd-e366-000000001670 |     TIMING | tripleo_ceph_run_ansible : run ceph-ansible | undercloud | 0:09:41.116734 | 561.98s                                                                         
2022-05-28 11:47:08.946081 | 525400cf-ccf8-4dfd-e366-000000001670 |     TIMING | tripleo_ceph_run_ansible : run ceph-ansible | undercloud | 0:09:41.333024 | 562.20s                                                                         
2022-05-28 11:47:09.076978 | 525400cf-ccf8-4dfd-e366-000000001671 |       TASK | search output of ceph-ansible run(s) non-zero return codes                                                                                                  
2022-05-28 11:47:09.944889 | 525400cf-ccf8-4dfd-e366-000000001671 |    SKIPPED | search output of ceph-ansible run(s) non-zero return codes | undercloud | result={                                                                          
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result",                                                                                                                              
    "changed": false                                                                                                                                                                                                                         
}                                                                                                                                                                                                                                            
2022-05-28 11:47:10.207290 | 525400cf-ccf8-4dfd-e366-000000001671 |     TIMING | tripleo_ceph_run_ansible : search output of ceph-ansible run(s) non-zero return codes | undercloud | 0:09:42.594229 | 1.13s                                 
2022-05-28 11:47:10.384937 | 525400cf-ccf8-4dfd-e366-000000001672 |       TASK | print ceph-ansible output in case of failure
2022-05-28 11:47:10.413530 | 525400cf-ccf8-4dfd-e366-000000001672 |    SKIPPED | print ceph-ansible output in case of failure | undercloud
2022-05-28 11:47:10.414043 | 525400cf-ccf8-4dfd-e366-000000001672 |     TIMING | tripleo_ceph_run_ansible : print ceph-ansible output in case of failure | undercloud | 0:09:42.800995 | 0.03s
2022-05-28 11:47:10.422072 | 525400cf-ccf8-4dfd-e366-0000000004e8 |       TASK | configure ceph clients
2022-05-28 11:47:10.439570 | 525400cf-ccf8-4dfd-e366-0000000004e8 |     TIMING | configure ceph clients | undercloud | 0:09:42.826515 | 0.02s
2022-05-28 11:47:10.508668 | 525400cf-ccf8-4dfd-e366-000000001922 |       TASK | Gather variables for each operating system
2022-05-28 11:47:10.537123 | 525400cf-ccf8-4dfd-e366-000000001922 |      FATAL | Gather variables for each operating system | undercloud | error={
    "msg": "No file was found when using first_found. Use errors='ignore' to allow this task to be skipped if no files are found"
}
2022-05-28 11:47:10.537568 | 525400cf-ccf8-4dfd-e366-000000001922 |     TIMING | tripleo_ceph_client : Gather variables for each operating system | undercloud | 0:09:42.924520 | 0.03s

NO MORE HOSTS LEFT ***************************************************************************************************************************************************************************************************************************

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
ceph-0                     : ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
ceph-1                     : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
ceph-2                     : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
compute-0                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
compute-1                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
controller-0               : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
controller-1               : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
controller-2               : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2   
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
undercloud                 : ok=75   changed=5    unreachable=0    failed=1    skipped=78   rescued=0    ignored=2   


2022-05-28 11:47:10.539809 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-28 11:47:10.539903 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 158        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-28 11:47:10.539990 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:09:42.926948 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-28 11:47:10.540084 |                                 UUID |       Info |       Host |   Task Name |   Run Time
2022-05-28 11:47:10.540163 | 525400cf-ccf8-4dfd-e366-000000001670 |    SUMMARY | undercloud | tripleo_ceph_run_ansible : run ceph-ansible | 562.20s
2022-05-28 11:47:10.540269 | 525400cf-ccf8-4dfd-e366-000000001426 |    SUMMARY | undercloud | tripleo_ceph_uuid : run nodes-uuid command | 3.31s
2022-05-28 11:47:10.540366 | 525400cf-ccf8-4dfd-e366-000000001662 |    SUMMARY | undercloud | tripleo_ceph_run_ansible : run create_ceph_ansible_remote_tmp command | 1.79s
2022-05-28 11:47:10.540454 | 525400cf-ccf8-4dfd-e366-000000001671 |    SUMMARY | undercloud | tripleo_ceph_run_ansible : search output of ceph-ansible run(s) non-zero return codes | 1.13s
2022-05-28 11:47:10.540536 | 525400cf-ccf8-4dfd-e366-000000000941 |    SUMMARY | undercloud | tripleo_ceph_work_dir : create ceph-ansible temp dirs | 0.84s
2022-05-28 11:47:10.540611 | 525400cf-ccf8-4dfd-e366-000000000eca |    SUMMARY | undercloud | tripleo_ceph_work_dir : create ceph-ansible temp dirs | 0.65s
2022-05-28 11:47:10.540696 | 525400cf-ccf8-4dfd-e366-000000000704 |    SUMMARY |  localhost | Gathering Facts | 0.60s
2022-05-28 11:47:10.540775 | 525400cf-ccf8-4dfd-e366-000000000957 |    SUMMARY | undercloud | tripleo_ceph_work_dir : generate ceph-ansible group vars all | 0.59s
2022-05-28 11:47:10.540863 | 525400cf-ccf8-4dfd-e366-000000000497 |    SUMMARY | undercloud | generate ceph-ansible group vars mgrs | 0.41s

Comment 5 John Fulton 2022-06-06 15:38:29 UTC
Please try again with the following:

parameter_defaults:
  CephConfigPath: "/etc/ceph"

We will need the documentation for the Ceph upgrade to include using the above.


Starting in 17 ceph client config is in /var/lib/tripleo-config/ceph but for older versions (and during adoption) it will be in the old location (/etc/ceph). 

  https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/cephadm/ceph-client.yaml#L58-L62

Comment 36 errata-xmlrpc 2023-08-16 01:11:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4577


Note You need to log in before you can comment on or make changes to this bug.