Bug 1702498
| Summary: | Re-running overcloud deploy command with Ceph nodes fails on TASK [unpack downloaded ceph-ansible fetch tarball to fetch directory | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Marius Cornea <mcornea> |
| Component: | openstack-tripleo-heat-templates | Assignee: | John Fulton <johfulto> |
| Status: | CLOSED ERRATA | QA Contact: | Yogev Rabl <yrabl> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 15.0 (Stein) | CC: | dbecker, emacchi, gcharot, gfidente, johfulto, jvisser, mburns, morazi, sclewis, tenobreg |
| Target Milestone: | beta | Keywords: | Triaged |
| Target Release: | 15.0 (Stein) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-tripleo-heat-templates-10.5.1-0.20190509160423.4dac4dc.el8ost.noarch | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-09-21 11:21:34 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1705248 | ||
| Bug Blocks: | |||
WORKAROUND: If you're using the defaults: source stackrc swift list swift list ceph_ansible_fetch_dir swift download ceph_ansible_fetch_dir <tar file from last command> If you're using LocalCephAnsibleFetchDirectoryBackup rm -rf /tmp/fetch_dir/temporary_dir.tar.gz Or remove whatever you set LocalCephAnsibleFetchDirectoryBackup to. E.g. the above example assumes it was set to /tmp/fetch_dir/. This should let you redeploy, but if you're scaling monitors the above actions will cause monitor scaling to not work. swift download ceph_ansible_fetch_dir <tar file from last command>(In reply to John Fulton from comment #1) > WORKAROUND: > > If you're using the defaults: > > source stackrc > swift list > swift list ceph_ansible_fetch_dir > swift download ceph_ansible_fetch_dir <tar file from last command> AND (and this step is important): swift delete ceph_ansible_fetch_dir <tar file from last command> Verified Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:2811 Clearing needinfo. No doctext necessary. |
Description of problem: Re-running overcloud deploy command with Ceph nodes fails on TASK [unpack downloaded ceph-ansible fetch tarball to fetch directory: 2019-04-23 22:56:10,754 p=21491 u=mistral | TASK [unpack downloaded ceph-ansible fetch tarball to fetch directory] ********* 2019-04-23 22:56:10,754 p=21491 u=mistral | Tuesday 23 April 2019 22:56:10 +0000 (0:00:00.114) 0:15:27.699 ********* 2019-04-23 22:56:11,066 p=21491 u=mistral | fatal: [undercloud]: FAILED! => {"changed": true, "cmd": "/usr/bin/gtar --gzip --extract --file /tmp/temporary_dir_old.tar.gz -C /var/lib/mistral/overcloud/ceph-ansible/fetch_dir", "delta": "0:00:00.008969", "end": "2019-04-23 22:56:11.043733", "msg": "non-zero return code", "rc": 2, "start": "2019-04-23 22:56:11.034764", "stderr": "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.mgr.controller-0.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.mgr.controller-1.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.mgr.controller-2.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.admin.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.openstack.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.manila.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.radosgw.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-osd: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rgw: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-mds: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd-mirror: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-osd/ceph.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rgw/ceph.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-mds/ceph.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd/ceph.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd-mirror/ceph.keyring: Cannot open: File exists\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph: Cannot utime: Operation not permitted\n/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib: Cannot utime: Operation not permitted\n/usr/bin/gtar: Exiting with failure status due to previous errors", "stderr_lines": ["/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.mgr.controller-0.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.mgr.controller-1.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.mgr.controller-2.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.admin.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.openstack.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.manila.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph/ceph.client.radosgw.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/etc/ceph: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-osd: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rgw: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-mds: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd-mirror: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-osd/ceph.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rgw/ceph.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-mds/ceph.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd/ceph.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph/bootstrap-rbd-mirror/ceph.keyring: Cannot open: File exists", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib/ceph: Cannot utime: Operation not permitted", "/usr/bin/gtar: 126e4394-6606-11e9-b9fb-5254007e41bb/var/lib: Cannot utime: Operation not permitted", "/usr/bin/gtar: Exiting with failure status due to previous errors"], "stdout": "", "stdout_lines": []} Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-10.5.1-0.20190423085106.3f148c4.el8ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy overcloud with ceph enabled 2. Re-run overcloud deploy with --config-download-only Actual results: Fails on TASK [unpack downloaded ceph-ansible fetch tarball to fetch directory Expected results: No failures on re-run. Additional info: