During the upgrade from OSP14 to OSP15, the overcloud upgrade prepare step seems to be passing. However, when taking a deeper look at the logs, we can observe the following ERROR log:

2019-07-30 10:07:04 | Connection reset by 192.168.24.9 port 22^M
2019-07-30 10:07:04 | Enabling ssh admin (tripleo-admin) for hosts:
2019-07-30 10:07:04 | 192.168.24.9 192.168.24.7 192.168.24.8 192.168.24.16
2019-07-30 10:07:04 | Using ssh user heat-admin for initial connection.
2019-07-30 10:07:04 | Using ssh key at /home/stack/.ssh/id_rsa for initial connection.
2019-07-30 10:07:04 | Inserting TripleO short term key for 192.168.24.9
2019-07-30 10:07:04 | Removing short term keys locally
2019-07-30 10:07:04 | 2019-07-30 10:07:04.988 51635 ERROR tripleoclient.v1.overcloud_upgrade.MajorUpgradePrepare [-] Couldn't not import keys to one of ['192.168.24.9', '192.168.24.7', '192.168.24.8', '192.168.24.16']. Check if the user/ip are corrects.
2019-07-30 10:07:04 | : subprocess.CalledProcessError: Command '['ssh', '-o', 'ConnectionAttempts=6', '-o', 'ConnectTimeout=30', '-o', 'StrictHostKeyChecking=no', '-o', 'PasswordAuthentication=no', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-i', '/home/stack/.ssh/id_rsa', '-l', 'heat-admin', '192.168.24.9', "echo -e '\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC40BQ4jyGYRbC1euZDmkvTjN4U0yuNaTLKHB4cgeH2FZDAY1dksEBoO4ba7LzCi33WRUGk11ojBz+Jwtwr9FUQlcnk/pKGVHOJVLxOTg7Hnz2Zit4y1vd3HtglRI5fWnDCMeQ1p/a1u6ypdMBHMaAf17tMfi8f94qlwXovS0PQkZo1Nn3fvLy9QM0e2gaCd0SZsOnXHiLy4dRtNXbxQA0ODOat92N3F8U3POuQMy1AlRXozBurL2C7nYiSKyAphWXB9b/KoxqA9J+ZWyjoPFa94YA6O0stBxTYMXNHHpgaKg172omU7FlLa707e6m5K3u2/cGdavdty6u2pPTcU7/+5ngGE3ucnPEHuVZwoyc0L31IuAhnpHdp3hGMc37oqfD609bx6upX86zAupMFO3W1NgMZ1Od1Bm5/VW4IfVc8Jnif/LmPBRS3XD94P+/7zI2UsOZPwkt61JaOBRDOe1FbK/WXuDS8vEUtKgnLXiWIm5dLxceWMcRo32egfx6UZ9h04oYftmWVZ1LPgCTgro9KExZhirEXlY3i8TnAaIsFqcgtENLIiOe0CilVjyrt0Y9Q6L06KH/zmvnwpQ8uVQiwG6DuwLEATvC4olAZ0f/02EqMVLDCuhO7W6SM9ttXtGYQCwqDn+fN3MbyGClOX/Bmpx+6KP0+PKU3v1JylnOu9Q== TripleO split stack short term key\n' >> $HOME/.ssh/authorized_keys"]' returned non-zero exit status 255.^[[00m
2019-07-30 10:07:04 | 2019-07-30 10:07:04.989 51635 INFO tripleoclient.v1.overcloud_upgrade.MajorUpgradePrepare [-] Completed Overcloud Upgrade Prepare for stack overcloud^[[00m
2019-07-30 10:07:04 | 2019-07-30 10:07:04.991 51635 INFO osc_lib.shell [-] END return value: None^[[00m

When checking if those IPs where accessible from the Undercloud they seemed to be pingable. Also from the mistral_executor container, which is the place where the Mistral  workflow is ran.

Besides this error log, some confusing logs are appearing at the very end of the overcloud upgrade prepare:

2019-07-30 10:07:04 | 2019-07-30 10:07:04.989 51635 INFO tripleoclient.v1.overcloud_upgrade.MajorUpgradePrepare [-] Completed Overcloud Upgrade Prepare for stack overcloud^[[00m
2019-07-30 10:07:04 | 2019-07-30 10:07:04.991 51635 INFO osc_lib.shell [-] END return value: None^[[00m
2019-07-30 10:07:04 | /usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py:225: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmptqbtjjum/id_rsa.pub' mode='r' encoding='UTF-8'>
2019-07-30 10:07:04 |   tmp_key_public_contents = open(tmp_key_public).read()
2019-07-30 10:07:04 | /usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py:170: ResourceWarning: unclosed <socket.socket fd=9, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('192.168.24.1', 34672), raddr=('192.168.24.9', 22)>
2019-07-30 10:07:04 |   socket.socket().connect((host, 22))
2019-07-30 10:07:05 | sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.24.2', 34048), raddr=('192.168.24.2', 13808)>
2019-07-30 10:07:05 | sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.24.2', 35242)>
2019-07-30 10:07:05 | sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.24.2', 53322)>
2019-07-30 10:07:05 | sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.24.2', 51414)>


These might be unrelated to this issue, so if required I could open another bug for them. However, I will be using this Bugzilla for tracking them and at least triaging the issue.

Reproduce:

Follow the steps to upgrade from OSP14 to OSP15 from the WIP guideline: https://gitlab.cee.redhat.com/osp15/osp-upgrade-el8/blob/master/README.md

The error will appear when checking the logs of the overcloud upgrade prepare command.