Description of problem: The instance creation fails with "exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8e22b053-ccf0-47dd-b968-93f61d4ee22e" The same volume on other compute nodes works. This issue happens for every volume from VMAX backend The issue could be bind with BUG 1942079 and 1964078. But I'm not sure. Version-Release number of selected component (if applicable): [redhat-release] Red Hat Enterprise Linux Server release 7.7 (Maipo) [rhosp-release] Red Hat OpenStack Platform release 13.0.10 (Queens) How reproducible: The customer can reproduce the error every time they try to create an instance from a volume. Steps to Reproduce: 1. create volume in VMAX backend from an image 2. Create an instance from that volume Actual results: Instance creation fails Expected results: instance creation succeed Additional info: From nova.log on cmpc23 we can see log about searching Multipath device: 2021-05-21 15:32:17.741 8 INFO os_brick.initiator.linuxscsi [req-95e3cfc2-7647-4c65-baa8-84ea9345c8a9 ad9a17ce8088485ca950656ba84345b5 ff9c71872c20417e83e4387fccf85434 - default default] Find Multipath device file for volume WWN 360000970000297802096533030373435 2021-05-21 15:32:17.742 8 DEBUG os_brick.initiator.linuxscsi [req-95e3cfc2-7647-4c65-baa8-84ea9345c8a9 ad9a17ce8088485ca950656ba84345b5 ff9c71872c20417e83e4387fccf85434 - default default] /dev/disk/by-id/dm-uuid-mpath-360000970000297802096533030373435 doesn't exists yet. wait_for_path /usr/lib/python2.7/site-packages/os_brick/initiator/linuxscsi.py:337 2021-05-21 15:32:17.742 8 DEBUG os_brick.utils [req-95e3cfc2-7647-4c65-baa8-84ea9345c8a9 ad9a17ce8088485ca950656ba84345b5 ff9c71872c20417e83e4387fccf85434 - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/os_brick/utils.py:45 2021-05-21 15:32:29.758 8 WARNING os_brick.initiator.linuxscsi [req-95e3cfc2-7647-4c65-baa8-84ea9345c8a9 ad9a17ce8088485ca950656ba84345b5 ff9c71872c20417e83e4387fccf85434 - default default] couldn't find a valid multipath device path for 360000970000297802096533030373435: VolumeDeviceNotFound: Volume device not found at /dev/mapper/360000970000297802096533030373435. 2021-05-21 15:32:29.826 8 WARNING os_brick.initiator.linuxscsi [req-95e3cfc2-7647-4c65-baa8-84ea9345c8a9 ad9a17ce8088485ca950656ba84345b5 ff9c71872c20417e83e4387fccf85434 - default default] multipath call failed exit 1: ProcessExecutionError: Unexpected error while running command. and then failed with Failed to execute command multipath -l /dev/sdl 2021-05-21 15:32:29.828 8 ERROR nova.compute.manager [req-95e3cfc2-7647-4c65-baa8-84ea9345c8a9 ad9a17ce8088485ca950656ba84345b5 ff9c71872c20417e83e4387fccf85434 - default default] [instance: 8e22b053-ccf0-47dd-b968-93f61d4ee22e] Instance failed to spawn: CommandExecutionFailed: Failed to execute command multipath -l /dev/sdl
OSP13 support officially ended on 27 June 2023