Bug 1821816
| Summary: | [OSP15->16] Converge step fails during ceph-ansible preparation | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Jose Luis Franco <jfrancoa> |
| Component: | tripleo-ansible | Assignee: | John Fulton <johfulto> |
| Status: | CLOSED ERRATA | QA Contact: | Yogev Rabl <yrabl> |
| Severity: | medium | Docs Contact: | |
| Priority: | high | ||
| Version: | 16.0 (Train) | CC: | johfulto, lbezdick, yrabl |
| Target Milestone: | beta | Keywords: | Triaged |
| Target Release: | 16.1 (Train on RHEL 8.2) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | tripleo-ansible-0.5.1-0.20200425093424.add11b7.el8ost.noarch.rpm | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-07-29 07:51:29 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
If this bug requires doc text for errata release, please set the 'Doc Type' and provide draft text according to the template in the 'Doc Text' field. The documentation team will review, edit, and approve the text. If this bug does not require doc text, please set the 'requires_doc_text' flag to '-'. (In reply to Alex McLeod from comment #5) > If this bug does not require doc text, please set the 'requires_doc_text' > flag to '-'. done Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3148 |
Description of problem: When upgrading from OSP15 to OSP16, at the time of running the converge step (executes the whole deployment underneath) the following task failed: 2020-04-07 14:40:05,075 p=48174 u=mistral | TASK [tripleo-ceph-common : set calling_ansible_environment_variables] ********* 2020-04-07 14:40:05,076 p=48174 u=mistral | Tuesday 07 April 2020 14:40:05 +0000 (0:00:00.134) 0:06:57.979 ********* 2020-04-07 14:40:05,134 p=48174 u=mistral | skipping: [undercloud] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-07 14:40:05,194 p=48174 u=mistral | TASK [create ceph-ansible working direcotry] *********************************** 2020-04-07 14:40:05,195 p=48174 u=mistral | Tuesday 07 April 2020 14:40:05 +0000 (0:00:00.118) 0:06:58.098 ********* 2020-04-07 14:40:05,470 p=48174 u=mistral | TASK [tripleo-ceph-work-dir : create ceph-ansible temp dirs] ******************* 2020-04-07 14:40:05,471 p=48174 u=mistral | Tuesday 07 April 2020 14:40:05 +0000 (0:00:00.275) 0:06:58.374 ********* 2020-04-07 14:40:05,832 p=48174 u=mistral | changed: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible) => {"ansible_loop_var": "item", "changed": true, "gid": 42430, "group": "42430", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible", "secontext": "system_u:object_r:container_file_t:s0", "size": 205, "state": "directory", "uid": 1002} 2020-04-07 14:40:06,137 p=48174 u=mistral | changed: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible/group_vars) => {"ansible_loop_var": "item", "changed": true, "gid": 42430, "group": "42430", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/group_vars", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/group_vars", "secontext": "system_u:object_r:container_file_t:s0", "size": 88, "state": "directory", "uid": 1002} 2020-04-07 14:40:06,390 p=48174 u=mistral | changed: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible/host_vars) => {"ansible_loop_var": "item", "changed": true, "gid": 42430, "group": "42430", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/host_vars", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/host_vars", "secontext": "system_u:object_r:container_file_t:s0", "size": 174, "state": "directory", "uid": 1002} 2020-04-07 14:40:06,646 p=48174 u=mistral | changed: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible/fetch_dir) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/fetch_dir", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/fetch_dir", "secontext": "unconfined_u:object_r:container_file_t:s0", "size": 6, "state": "directory", "uid": 1002} 2020-04-07 14:40:06,729 p=48174 u=mistral | TASK [tripleo-ceph-work-dir : symbolic link to tripleo inventory from ceph-ansible work directory] *** 2020-04-07 14:40:06,729 p=48174 u=mistral | Tuesday 07 April 2020 14:40:06 +0000 (0:00:01.258) 0:06:59.632 ********* 2020-04-07 14:40:07,040 p=48174 u=mistral | fatal: [undercloud]: FAILED! => {"changed": false, "gid": 42430, "group": "42430", "mode": "0664", "msg": "refusing to convert from file to symlink for /var/lib/mistral/qe-Cloud-0/ceph-ansible/inventory.yml", "owner": "42430", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/inventory.yml", "secontext": "system_u:object_r:container_file_t:s0", "size": 1303, "state": "file", "uid": 42430} 2020-04-07 14:40:07,041 p=48174 u=mistral | NO MORE HOSTS LEFT ************************************************************* 2020-04-07 14:40:07,042 p=48174 u=mistral | PLAY RECAP ********************************************************************* 2020-04-07 14:40:07,042 p=48174 u=mistral | ceph-0 : ok=103 changed=21 unreachable=0 failed=0 skipped=55 rescued=0 ignored=0 2020-04-07 14:40:07,043 p=48174 u=mistral | ceph-1 : ok=99 changed=21 unreachable=0 failed=0 skipped=55 rescued=0 ignored=0 2020-04-07 14:40:07,043 p=48174 u=mistral | ceph-2 : ok=99 changed=21 unreachable=0 failed=0 skipped=55 rescued=0 ignored=0 2020-04-07 14:40:07,043 p=48174 u=mistral | compute-0 : ok=128 changed=37 unreachable=0 failed=0 skipped=71 rescued=0 ignored=0 2020-04-07 14:40:07,043 p=48174 u=mistral | compute-1 : ok=127 changed=37 unreachable=0 failed=0 skipped=71 rescued=0 ignored=0 2020-04-07 14:40:07,043 p=48174 u=mistral | controller-0 : ok=151 changed=51 unreachable=0 failed=0 skipped=73 rescued=0 ignored=0 2020-04-07 14:40:07,043 p=48174 u=mistral | controller-1 : ok=148 changed=50 unreachable=0 failed=0 skipped=73 rescued=0 ignored=0 2020-04-07 14:40:07,043 p=48174 u=mistral | controller-2 : ok=148 changed=50 unreachable=0 failed=0 skipped=73 rescued=0 ignored=0 2020-04-07 14:40:07,044 p=48174 u=mistral | undercloud : ok=12 changed=3 unreachable=0 failed=1 skipped=4 rescued=0 ignored=0 2020-04-07 14:40:07,044 p=48174 u=mistral | Tuesday 07 April 2020 14:40:07 +0000 (0:00:00.314) 0:06:59.947 ********* 2020-04-07 14:40:07,044 p=48174 u=mistral | =============================================================================== After some debugging and local testing, I did try adding force: yes to the task: - name: symbolic link to tripleo inventory from ceph-ansible work directory # If we call ceph-ansible with the same inventory as the calling # playbook, then config-download/groups_vars will be used instead # of config-download/ceph-ansible/group_vars. file: src: "{{ inventory_file }}" dest: "{{ playbook_dir }}/ceph-ansible/inventory.yml" state: link force: true tags: - run_uuid_ansible - run_ceph_ansible Re-ran the same converge and then the task successfuly executed: TASK [tripleo-ceph-work-dir : create ceph-ansible temp dirs] ******************* Tuesday 07 April 2020 16:14:32 +0000 (0:00:00.254) 0:07:43.964 ********* ok: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible) => {"ansible_loop_var": "item", "changed": false, "gid": 42430, "group": "42430", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible", "secontext": "system_u:object_r:container_file_t:s0", "size": 222, "state": "directory", "uid": 1002} ok: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible/group_vars) => {"ansible_loop_var": "item", "changed": false, "gid": 42430, "group": "42430", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/group_vars", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/group_vars", "secontext": "system_u:object_r:container_file_t:s0", "size": 88, "state": "directory", "uid": 1002} ok: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible/host_vars) => {"ansible_loop_var": "item", "changed": false, "gid": 42430, "group": "42430", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/host_vars", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/host_vars", "secontext": "system_u:object_r:container_file_t:s0", "size": 174, "state": "directory", "uid": 1002} ok: [undercloud] => (item=/var/lib/mistral/qe-Cloud-0/ceph-ansible/fetch_dir) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/fetch_dir", "mode": "0755", "owner": "tripleo-admin", "path": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/fetch_dir", "secontext": "unconfined_u:object_r:container_file_t:s0", "size": 6, "state": "directory", "uid": 1002} TASK [tripleo-ceph-work-dir : symbolic link to tripleo inventory from ceph-ansible work directory] *** Tuesday 07 April 2020 16:14:33 +0000 (0:00:01.312) 0:07:45.277 ********* changed: [undercloud] => {"changed": true, "dest": "/var/lib/mistral/qe-Cloud-0/ceph-ansible/inventory.yml", "gid": 1003, "group": "tripleo-admin", "mode": "0777", "owner": "tripleo-admin", "secontext": "unconfined_u:object_r:container_file_t:s0", "size": 58, "src": "/var/lib/mistral/qe-Cloud-0/tripleo-ansible-inventory.yaml", "state": "link", "uid": 1002} Checking the upstreame code, it looks like that line was missed to backport to tripleo-ansible stable/train: https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_ceph_work_dir/tasks/prepare.yml#L37 Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: