Bug 1312187
| Summary: | Ironic fails to find the disk to write image to | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Sai Sindhur Malleni <smalleni> |
| Component: | openstack-ironic-python-agent | Assignee: | Dmitry Tantsur <dtantsur> |
| Status: | CLOSED ERRATA | QA Contact: | Raviv Bar-Tal <rbartal> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 8.0 (Liberty) | CC: | augol, chris.brown, jschluet, mburns, rhel-osp-director-maint, slinaber, smalleni, yeylon |
| Target Milestone: | ga | ||
| Target Release: | 8.0 (Liberty) | ||
| Hardware: | x86_64 | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-ironic-python-agent-1.1.0-6.el7ost | Doc Type: | Bug Fix |
| Doc Text: |
Sometimes, hard drives were not available in time for a deployment ramdisk run. Consequently, the deployment failed if the ramdisk was unable to find the required root device. With this update, the "udev settle" command is executed before enumerating disks in the ramdisk, and the deployment no longer fails due to the missing root device.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-04-07 21:31:26 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
So I've tested the ramdisk and apparently the problem is just the time that the IPA service starts. I've first tried to modify the openstack-ironic-python-agent.service in the ramdisk to Require the systemd-udev-settle.service [0] service, but that didn't work. Modifying the ramdisk manually to trigger udevadm and then settle in the code actually worked for me, e.g: # udevadm trigger --verbose --dry-run --type=devices --subsystem-match=scsi_disk # udevadm settle I will work on a patch for IPA to do it. [0] https://github.com/systemd/systemd/blob/master/units/systemd-udev-settle.service.in Fix was approved upstream and backported [0] [0] https://code.engineering.redhat.com/gerrit/#/c/68938/ Hi Sindhur, I don't really know how to verify this bug, There is not enough information and hw specification. Can you tell if the fix did solve the problem for you? If so I'll verify we have the current package in the release and close this bug. Thanks Raviv Bar-Tal, I tried the latest puddle/poodle and did not hit the issue, in fact in the performance team we have several working OSP 8 deployments, and haven't seen this issue after the fix. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0603.html |
Description of problem: Overcloud deploy always fails after retries timing out. In a 3 controller+2 compute deployment at the most 4 of the nodes deployed before the stack create fails. Each time the nodes that fail to deploy are not necessarilythe same. Introspection completes without any complaints but in the ironic debug logs the following error is seen frequently. Some nodes which show the error also go through installation in subsequent attempts but some fail after retries. Failed to deploy instance: Failed to start the iSCSI target to deploy the node 79aa5db0-df3a-4f2e-9e6b-45820b18a97c. Error: {u'message': u'Error finding the disk or partition device to deploy the image onto: No suitable device was found for deployment - root device hints were not provided and all found block devices are smaller than 4294967296B [].', u'code': 404, u'type': u'DeviceNotFound', u'details': u'No suitable device was found for deployment - root device hints were not provided and all found block devices are smaller than 4294967296B [].'} The systems are PowerEdge R610 with 500gigs hard disk and as said introspection doesnt fail. How reproducible: Always on the environment I tried Steps to Reproduce: 1. Build undercloud 2.Finish introspection 3. Deploy overcloud Actual results:Overcloud deploys Expected results: Deploy fails Additional info: Using the latest OSP 8 bits. Local_boot option wasn't used