Description of problem: Working on bz#1904681 we found that 1. We dump and compress database on controller-1 2. We transfer it to undercloud from controller-1, 3. We transfer it from undercloud to controller-0, unpack it and start database. Usually our undercloud is pretty small 10-50GB. However, we have clients that have openstack DB larger than 1TB. It means that transfer will never happen. How reproducible: Always Steps to Reproduce: 1. Create uc as 10Gb as OSP13 2. Create overcloud DB as 100Gb and try to run FFWD as OSP13 3. Run FFWD Actual results: Fail Expected results: Pass Additional info: Instead of doing transfer to undercloud we should initiate transfer from controller-1 to controller-0 directly which has enough space after leapp (consider it fresh system). In that case, transfer will be fast as we transfer 1 time instead of 2 times and transfer won't depend on diskspace of undercloud. I did a quick test and was able to transfer file with - hosts: controller-0 tasks: - name: Copy DB from Controller-1 to Controller-0 synchronize: src=~/DB.sql dest=~/DB.sql delegate_to: controller-1 In this case, file is transferred without copying to/from undercloud.
*** Bug 1904681 has been marked as a duplicate of this bug. ***
Validated in: https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/DFG/view/upgrades/view/ffu/job/DFG-upgrades-ffu-ffu-upgrade-13-16.1_director-rhel-virthost-3cont_2comp_3ceph-ipv4-vxlan-HA/140 http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-upgrades-ffu-ffu-upgrade-13-16.1_director-rhel-virthost-3cont_2comp_3ceph-ipv4-vxlan-HA/140/undercloud-0/home/stack/overcloud_upgrade_run-ceph-0,controller-0.log.gz 2021-05-04 13:35:51 | TASK [Transfer mysql data] ***************************************************** 2021-05-04 13:35:51 | Tuesday 04 May 2021 13:35:35 +0000 (0:00:00.062) 0:02:59.617 *********** 2021-05-04 13:35:51 | 2021-05-04 13:35:51 | TASK [tripleo-transfer : install requirements in src and dest hosts] *********** 2021-05-04 13:35:51 | Tuesday 04 May 2021 13:35:35 +0000 (0:00:00.173) 0:02:59.790 *********** 2021-05-04 13:35:51 | ok: [undercloud -> 192.168.24.22] => (item=controller-1) => {"ansible_loop_var": "item", "changed": false, "item": "controller-1", "msg": "", "rc": 0, "results": ["rsync-3.1.2-10.el7.x86_64 providing rsync is already installed", "openssh-clients-7.4p1-21.el7.x86_64 providing openssh-clients is already installed"]} 2021-05-04 13:35:51 | ok: [undercloud -> 192.168.24.24] => (item=controller-0) => {"ansible_loop_var": "item", "changed": false, "item": "controller-0", "msg": "Nothing to do", "rc": 0, "results": []} 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | TASK [tripleo-transfer : generate ssh key-pair in source host] ***************** 2021-05-04 13:36:08 | Tuesday 04 May 2021 13:35:51 +0000 (0:00:15.927) 0:03:15.717 *********** 2021-05-04 13:36:08 | changed: [undercloud -> 192.168.24.22] => {"changed": true, "comment": "", "filename": "/root/transfer_key", "fingerprint": "SHA256:p3DPYEcb/c1z7Sh+S/F4dYRaU1P0ADTJfMrhMgwYBeg", "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQGgo3YKO/RIYAnSNjir1gxbwX+L5y+6VLu6t0b0LWTfG+PJMS/QeDG7ysLFBSly9sandJlkHi+vvCnauFBvirc35e+V+ShG+t5fEtcpSOaNxlZoilEvw8M/IyDQeI2PTeMg7+qc2sMZJI5EOOHF6VtUXnKcxoZgBE4USpWlv9Kujl46g4y63wicazIG22Lbaeku1jSzrmyXw+gX19rQd09p6q7MANjwfZ/1T9xn0mkwOxlkJVG9M801ZI5Atm45erPFdVX427h1FbUSigYFDsbcqOiBaUhhtuhbO3YqXJEh66dfacjEMpMoangdSwP0Ph1sQvXy/IT0T0vCM6noFp6dZMRfZ41YebNwo9pKb0ATvT80IsLBJ6acD5P9q2wfhYbDcVvL5ZLZ1sgdTzlBuwP4PKcWXgQEQx9SPBNO2U7VI0LIWBh0oOo0ZszlFTcnnOpHNLB/2e4upmXuPkXp9kSDA5S2hRuo/uPnohSwZWs94btSBRRXLbtNWN3MYqCHLx2pNySGub7GU3OQ2hSHl40JTsVDG3zGuqsrQawDRSbkrzf6rImVWhh+sm8lm5AoCtB5B518YmxxowwfKuRUEIOPhQFfFrT4nDCvDzsX5Bylg9/o6yb4HmAvhcKSYPfHOcQoODS/C8Zx54BtQ1q2ZwV56tYfYoUcyatzM5uweTbw==", "size": 4096, "type": "rsa"} 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | TASK [tripleo-transfer : set authorized-keys in destination host] ************** 2021-05-04 13:36:08 | Tuesday 04 May 2021 13:35:52 +0000 (0:00:00.947) 0:03:16.665 *********** 2021-05-04 13:36:08 | changed: [undercloud -> 192.168.24.24] => {"changed": true, "comment": "Added by tripleo-transfer", "exclusive": false, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQGgo3YKO/RIYAnSNjir1gxbwX+L5y+6VLu6t0b0LWTfG+PJMS/QeDG7ysLFBSly9sandJlkHi+vvCnauFBvirc35e+V+ShG+t5fEtcpSOaNxlZoilEvw8M/IyDQeI2PTeMg7+qc2sMZJI5EOOHF6VtUXnKcxoZgBE4USpWlv9Kujl46g4y63wicazIG22Lbaeku1jSzrmyXw+gX19rQd09p6q7MANjwfZ/1T9xn0mkwOxlkJVG9M801ZI5Atm45erPFdVX427h1FbUSigYFDsbcqOiBaUhhtuhbO3YqXJEh66dfacjEMpMoangdSwP0Ph1sQvXy/IT0T0vCM6noFp6dZMRfZ41YebNwo9pKb0ATvT80IsLBJ6acD5P9q2wfhYbDcVvL5ZLZ1sgdTzlBuwP4PKcWXgQEQx9SPBNO2U7VI0LIWBh0oOo0ZszlFTcnnOpHNLB/2e4upmXuPkXp9kSDA5S2hRuo/uPnohSwZWs94btSBRRXLbtNWN3MYqCHLx2pNySGub7GU3OQ2hSHl40JTsVDG3zGuqsrQawDRSbkrzf6rImVWhh+sm8lm5AoCtB5B518YmxxowwfKuRUEIOPhQFfFrT4nDCvDzsX5Bylg9/o6yb4HmAvhcKSYPfHOcQoODS/C8Zx54BtQ1q2ZwV56tYfYoUcyatzM5uweTbw==", "key_options": null, "keyfile": "/home/tripleo-admin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tripleo-admin", "validate_certs": true} 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | TASK [tripleo-transfer : check flag file existence in destination host] ******** 2021-05-04 13:36:08 | Tuesday 04 May 2021 13:35:53 +0000 (0:00:00.744) 0:03:17.409 *********** 2021-05-04 13:36:08 | ok: [undercloud -> 192.168.24.24] => {"changed": false, "stat": {"exists": false}} 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | TASK [tripleo-transfer : fail if flag file exists] ***************************** 2021-05-04 13:36:08 | Tuesday 04 May 2021 13:35:53 +0000 (0:00:00.594) 0:03:18.004 *********** 2021-05-04 13:36:08 | skipping: [undercloud] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | TASK [tripleo-transfer : ensure directory for flag file exists] **************** 2021-05-04 13:36:08 | Tuesday 04 May 2021 13:35:53 +0000 (0:00:00.094) 0:03:18.099 *********** 2021-05-04 13:36:08 | changed: [undercloud -> 192.168.24.24] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/tripleo/transfer-flags", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 6, "state": "directory", "uid": 0} 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | TASK [tripleo-transfer : create the flag file] ********************************* 2021-05-04 13:36:08 | Tuesday 04 May 2021 13:35:54 +0000 (0:00:00.595) 0:03:18.695 *********** 2021-05-04 13:36:08 | changed: [undercloud -> 192.168.24.24] => {"changed": true, "dest": "/var/lib/tripleo/transfer-flags/var-lib-mysql", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 0, "state": "file", "uid": 0} 2021-05-04 13:36:08 | 2021-05-04 13:36:08 | TASK [tripleo-transfer : synchronize both directories] ************************* 2021-05-04 13:36:08 | Tuesday 04 May 2021 13:35:54 +0000 (0:00:00.399) 0:03:19.094 *********** 2021-05-04 13:36:08 | changed: [undercloud -> 192.168.24.22] => {"changed": true, "cmd": "/usr/bin/rsync -v --delay-updates -F --ignore-times --compress --archive --delete --rsync-path='sudo rsync' --rsh='ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/transfer_key' /var/lib/mysql/ tripleo-admin.24.24:/var/lib/mysql", "delta": "0:00:13.465667", "end": "2021-05-04 13:36:08.538203", "rc": 0, "start": "2021-05-04 13:35:55.072536", "stderr": "Warning: Permanently added '192.168.24.24' (ECDSA) to the list of known hosts.", "stderr_lines": ["Warning: Permanently added '192.168.24.24' (ECDSA) to the list of known hosts."], "stdout": "building file list ... done\n./\naria_log.00000001\naria_log_control\ngalera.cache\ngrastate.dat\nib_logfile0\nib_logfile1\nibdata1\nmulti-master.info\naodh/alarm.frm\naodh ...................................................................................................... "panko/trait_int.frm", "panko/trait_int.ibd", "panko/trait_text.frm", "panko/trait_text.ibd", "performance_schema/", "performance_schema/db.opt", "", "sent 26,566,966 bytes received 1,007,006 bytes 2,042,516.44 bytes/sec", "total size is 425,033,003 speedup is 15.41"]} 2021-05-04 13:36:11 | 2021-05-04 13:36:11 | 2021-05-04 13:36:11 | TASK [tripleo-transfer : clean up keys in source host] ************************* 2021-05-04 13:36:11 | Tuesday 04 May 2021 13:36:08 +0000 (0:00:13.948) 0:03:33.044 *********** 2021-05-04 13:36:11 | changed: [undercloud -> 192.168.24.22] => (item=~/transfer_key) => {"ansible_loop_var": "item", "changed": true, "item": "~/transfer_key", "path": "/root/transfer_key", "state": "absent"} 2021-05-04 13:36:11 | changed: [undercloud -> 192.168.24.22] => (item=~/transfer_key.pub) => {"ansible_loop_var": "item", "changed": true, "item": "~/transfer_key.pub", "path": "/root/transfer_key.pub", "state": "absent"} 2021-05-04 13:36:11 | 2021-05-04 13:36:11 | TASK [tripleo-transfer : remove public key from authorized keys in destination host] *** 2021-05-04 13:36:11 | Tuesday 04 May 2021 13:36:09 +0000 (0:00:00.650) 0:03:33.694 *********** 2021-05-04 13:36:11 | changed: [undercloud -> 192.168.24.24] => {"backup": "", "changed": true, "found": 1, "msg": "1 line(s) removed"} PACKAGES: http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-upgrades-ffu-ffu-upgrade-13-16.1_director-rhel-virthost-3cont_2comp_3ceph-ipv4-vxlan-HA/140/undercloud-0/var/log/dnf.rpm.log.gz 2021-05-04T11:48:15Z SUBDEBUG Installed: tripleo-ansible-0.5.1-1.20210323173506.el8ost.noarch 2021-05-04T11:48:15Z SUBDEBUG Installed: openstack-tripleo-heat-templates-11.3.2-1.20210408163452.el8ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenStack Platform 16.1.6 (tripleo-ansible) security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2119