Description of problem: Overcloud deploy always fails after retries timing out. In a 3 controller+2 compute deployment at the most 4 of the nodes deployed before the stack create fails. Each time the nodes that fail to deploy are not necessarilythe same. Introspection completes without any complaints but in the ironic debug logs the following error is seen frequently. Some nodes which show the error also go through installation in subsequent attempts but some fail after retries. Failed to deploy instance: Failed to start the iSCSI target to deploy the node 79aa5db0-df3a-4f2e-9e6b-45820b18a97c. Error: {u'message': u'Error finding the disk or partition device to deploy the image onto: No suitable device was found for deployment - root device hints were not provided and all found block devices are smaller than 4294967296B [].', u'code': 404, u'type': u'DeviceNotFound', u'details': u'No suitable device was found for deployment - root device hints were not provided and all found block devices are smaller than 4294967296B [].'} The systems are PowerEdge R610 with 500gigs hard disk and as said introspection doesnt fail. How reproducible: Always on the environment I tried Steps to Reproduce: 1. Build undercloud 2.Finish introspection 3. Deploy overcloud Actual results:Overcloud deploys Expected results: Deploy fails Additional info: Using the latest OSP 8 bits. Local_boot option wasn't used
So I've tested the ramdisk and apparently the problem is just the time that the IPA service starts. I've first tried to modify the openstack-ironic-python-agent.service in the ramdisk to Require the systemd-udev-settle.service [0] service, but that didn't work. Modifying the ramdisk manually to trigger udevadm and then settle in the code actually worked for me, e.g: # udevadm trigger --verbose --dry-run --type=devices --subsystem-match=scsi_disk # udevadm settle I will work on a patch for IPA to do it. [0] https://github.com/systemd/systemd/blob/master/units/systemd-udev-settle.service.in
Fix was approved upstream and backported [0] [0] https://code.engineering.redhat.com/gerrit/#/c/68938/
Hi Sindhur, I don't really know how to verify this bug, There is not enough information and hw specification. Can you tell if the fix did solve the problem for you? If so I'll verify we have the current package in the release and close this bug. Thanks
Raviv Bar-Tal, I tried the latest puddle/poodle and did not hit the issue, in fact in the performance team we have several working OSP 8 deployments, and haven't seen this issue after the fix.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0603.html