Bug 1263860
| Summary: | Not able to introspect all nodes when doing a bulk introspection on 70+ nodes | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | bigswitch <rhosp-bugs-internal> |
| Component: | rhosp-director | Assignee: | Dmitry Tantsur <dtantsur> |
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Shai Revivo <srevivo> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.0 (Kilo) | CC: | mburns, rhel-osp-director-maint, rhosp-bugs-internal |
| Target Milestone: | --- | ||
| Target Release: | 10.0 (Newton) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-08-22 11:11:49 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
*** Bug 1263861 has been marked as a duplicate of this bug. *** We may need to have our engineering group work with you in your lab environment, to ensure we are able to reproduce with the same hardware. Hi Chris, I have deployed about 30+ nodes from this, would re-running bulk introspection affect the environment? Rebooting those nodes is fine, just dont want to lose the openstack config on it. I am available tomorrow Sept 18 from 10am US PST. Please let me know if it works for your engineer. Hi! Rerunning introspection won't affect working nodes, but I need to double-check that Ironic will allow it at all. As to time, 10am PST seems to be 7pm for me, but I'll try to be available for a couple of hours. Hi Dmitry Sorry tomorrow doesnt work for me. Can we reschedule to sometime next week? I'll post another timeslot later Thanks Song Hi! Yeah, no problem. I'll be busy on Monday at 10am PST, other days seem to be free. Please try to schedule it in your morning, as I'm in Europe. Alternatively, we can grab someone else to help you. In the meanwhile, could you post the full output of sudo journalctl -u openstack-ironic-discoverd -u openstack-ironic-discoverd-dnsmasq ? I'll have a look at it in advance then. Hi Dmitry The default root partition of 50gig isnt big enough to hold the log files, and I have moved the log files to /home directory. unfortunately that mess up the services and I am planning to reinstall the undercloud. I've copied out all files in /var/log and zip it to https://bigswitch.box.com/s/8yc6u8ca7iq9s91vfqven9cwda1zhnag I'm not sure if it has the information you are looking for though. Unfortunately, the sosreport does not include logs from discoverd, probably because it only contains logs from 3 days. I'll try to figure out something based on other logs, but it's very likely that I will need discoverd logs (see command in comment #9). Hi! Any updates on this issue? At least having full logs would be nice, it's too hard to figure out anything based on logs from other services. This bug did not make the OSP 8.0 release. It is being deferred to OSP 10. Hello! Please reopen if you get updated data for this issue. Thanks! The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |
Description of problem: import 70+ nodes into openstack baremetal, and do a bulk introspection. After about 3 hours, not all nodes are discovered. some nodes are timing out subsequently doing a node by node introspection of the failed nodes work [stack@dell-undercloud ~]$ ironic node-list +--------------------------------------+------+---------------+-------------+-----------------+-------------+ | UUID | Name | Instance UUID | Power State | Provision State | Maintenance | +--------------------------------------+------+---------------+-------------+-----------------+-------------+ | c7fdba2d-8177-4729-bdd7-5b2aedb58044 | None | None | power off | available | False | | 8e3eeef0-5bf8-48fa-86ae-89706bcc3cd1 | None | None | power off | available | False | | 73a75b9f-f4f1-4dd0-af62-9237cc2779d6 | None | None | power off | available | False | | c7526e61-fe94-4857-bb39-33fb604de908 | None | None | power off | available | False | | 389f6cab-f712-4a9a-86f3-c3c7e1d14076 | None | None | power off | available | False | | 98ae6b68-cb01-4bf2-b8df-8ce1dc0b2de5 | None | None | power off | available | False | | 26554c4a-8318-4371-a11d-a18c5f7f9bf7 | None | None | power off | available | False | | beec77e1-bbd5-4c13-b5cf-5014779a3f6a | None | None | power off | available | False | | f50d37a6-ad5f-418f-8f9c-a2e480f2d442 | None | None | power off | available | False | | 9ed44703-8f39-4057-ace1-950415cde3eb | None | None | power off | available | False | | 42105127-7526-4cd6-a5dd-ca41a4268e6a | None | None | power off | available | False | | ed0b0f12-710c-4c3d-96b1-fcc1a5fc36a8 | None | None | power off | available | False | | d259b087-0bab-455f-86ee-036fb1f23e5f | None | None | power off | available | False | | 4e866cef-fb3c-4935-8f30-d0993e2a139c | None | None | power off | available | False | | 9fc5350f-df33-4b1b-bfbf-735c11f202ab | None | None | power off | available | False | | 3859a0eb-4b10-483f-978b-d73f3b8c162a | None | None | power off | available | False | | 3d057044-1939-4ef8-ae61-c985bbb919f7 | None | None | power off | available | False | | daf9009b-1a64-41ab-9a7f-a302407efdb9 | None | None | power off | available | False | | b8abced9-eb33-4313-b6c6-79e3b4f4f1b3 | None | None | power off | available | False | | 712ec08c-67d6-4d42-baa2-39d883b226c6 | None | None | power off | available | False | | 769786b0-fef1-4859-b784-0ca25969b75a | None | None | power off | available | False | | 804d63e8-d99e-40cf-8ea2-872df80ad28a | None | None | power off | available | False | | 20a2fe3a-630d-41c4-bba5-e24bb22a5fb0 | None | None | power off | available | False | | 790f1f0f-6efe-4c08-b710-03ce18ae912d | None | None | power off | available | False | | ef625302-86f5-4af4-ad38-6414d4b268fc | None | None | power off | available | False | | 009d0e77-c355-4904-952d-35f068725c20 | None | None | power off | available | False | | a7bf099a-208f-4746-a19a-631cccaee46f | None | None | power off | available | False | | fe4256fe-398b-46c6-964f-a0e1bd06fbd3 | None | None | power off | available | False | | 7ac0d4ec-a4a8-438b-8b2f-5f817a1a70cd | None | None | power off | available | False | | 1554cfe9-28c5-405f-9a28-7cbc71c8053f | None | None | power off | available | False | | 4795e1b0-0708-4be2-99de-412bd5401065 | None | None | power off | available | False | | 429aacc5-3476-40ff-86fd-9c7858c1b1d0 | None | None | power off | available | False | | 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa | None | None | power off | available | False | | 76e0f972-2a8b-4a74-916b-edcdca9b28d7 | None | None | power off | available | False | | 98140410-0311-407a-88c9-ce6d3bb9c32f | None | None | power off | available | False | | 8885598e-b9b8-4a16-9a19-a1dee15ddd44 | None | None | power off | available | False | | 0e26c0eb-92fa-4ab8-a13f-449653a6e19b | None | None | power off | available | False | | 1f0fd18e-2de1-4d12-8fd3-117c6b1968d1 | None | None | power off | available | False | | 560c181d-8da6-4997-9f5a-d899a6860cf5 | None | None | power off | available | False | | 372f7b66-9cf8-44c4-9136-b6554ed60e29 | None | None | power off | available | False | | 217323a6-614a-47db-90d0-7f00069d7038 | None | None | power off | available | False | | 71c8b4c6-7843-414a-8d3d-3f94f0d45089 | None | None | power off | available | False | | a736c564-a8bc-451f-996e-dd709551f45a | None | None | power off | available | False | | a0ef8d26-d61e-44cc-a058-764192663500 | None | None | power off | available | False | | 8dadaec4-2bc6-4f88-b205-5cdb016d0365 | None | None | power off | available | False | | 85fa62b6-10cd-4249-aaa3-cabfa3e85751 | None | None | power off | available | False | | eb9a4e05-6fc9-434b-9f33-6a31208dd888 | None | None | power off | available | False | | 4b819e73-4b3c-4746-a360-d4e2d6cb2c97 | None | None | power off | available | False | | 82e7ecfa-7a73-40e9-8ae3-55ba3b467ee8 | None | None | power off | available | False | | 391e5097-8cd7-40ac-8f2d-9f80adaf76ec | None | None | power off | available | False | | 29773e1b-05d6-4f2b-98e1-981f84b217ec | None | None | power off | available | False | | 40b034bc-0c99-4881-be8f-4e4ff34bce69 | None | None | power off | available | False | | 9658eb40-69be-4d79-b10d-15eee12bfd7a | None | None | power off | available | False | | 30b7480a-ff06-4510-8282-6d7c37073a49 | None | None | power off | available | False | | 216632a7-5012-4eb8-8117-2a4ac5245673 | None | None | power off | available | False | | 523d8a18-5f10-4c3e-82ff-c4ada7e06dba | None | None | power off | available | False | | aa8bc63c-0a3d-4924-9cc6-a81813126e07 | None | None | power off | available | False | | c274afdb-efc7-423b-abbd-2bb836ff1f1c | None | None | power off | available | False | | 9452b455-e67e-4fc8-b81b-0bc1ee7c97e8 | None | None | power off | available | False | | 503ae852-1dff-4bec-8ae6-0f9bcbd0d322 | None | None | power off | available | False | | 9d4be85a-40eb-41c1-a008-5922e39e865d | None | None | power off | available | False | | be7506a3-3211-4606-b1dd-20d822dfb4c0 | None | None | power off | available | False | | 011f6b22-059c-42c5-9e84-15b3c074d455 | None | None | power off | available | False | | cfb3ea0d-2f49-4bd0-aa7b-ab72a46438c0 | None | None | power off | available | False | | ff28c010-2d5b-4cc7-b111-270908a18e1f | None | None | power off | available | False | | 1da69901-7a14-4393-a61c-be02347039ec | None | None | power off | available | False | | 90b14609-c8ef-45bb-b173-0af540e3183f | None | None | power off | available | False | | 18c6a449-68f3-4b2d-93f5-cdfaa12ceeeb | None | None | power off | available | False | | ebbd548f-091c-410a-9e8f-7e783e947734 | None | None | power off | available | False | | 64dfd30c-50ec-41f1-9aa8-498ac3b8026a | None | None | power off | available | False | | 3c2abe43-42c4-4689-b0df-3d02d116bbe1 | None | None | power off | available | False | +--------------------------------------+------+---------------+-------------+-----------------+-------------+ [stack@dell-undercloud ~]$ openstack baremetal introspection bulk start Setting available nodes to manageable... Starting introspection of node: c7fdba2d-8177-4729-bdd7-5b2aedb58044 Starting introspection of node: 8e3eeef0-5bf8-48fa-86ae-89706bcc3cd1 Starting introspection of node: 73a75b9f-f4f1-4dd0-af62-9237cc2779d6 Starting introspection of node: c7526e61-fe94-4857-bb39-33fb604de908 Starting introspection of node: 389f6cab-f712-4a9a-86f3-c3c7e1d14076 Starting introspection of node: 98ae6b68-cb01-4bf2-b8df-8ce1dc0b2de5 Starting introspection of node: 26554c4a-8318-4371-a11d-a18c5f7f9bf7 Starting introspection of node: beec77e1-bbd5-4c13-b5cf-5014779a3f6a Starting introspection of node: f50d37a6-ad5f-418f-8f9c-a2e480f2d442 Starting introspection of node: 9ed44703-8f39-4057-ace1-950415cde3eb Starting introspection of node: 42105127-7526-4cd6-a5dd-ca41a4268e6a Starting introspection of node: ed0b0f12-710c-4c3d-96b1-fcc1a5fc36a8 Starting introspection of node: d259b087-0bab-455f-86ee-036fb1f23e5f Starting introspection of node: 4e866cef-fb3c-4935-8f30-d0993e2a139c Starting introspection of node: 9fc5350f-df33-4b1b-bfbf-735c11f202ab Starting introspection of node: 3859a0eb-4b10-483f-978b-d73f3b8c162a Starting introspection of node: 3d057044-1939-4ef8-ae61-c985bbb919f7 Starting introspection of node: daf9009b-1a64-41ab-9a7f-a302407efdb9 Starting introspection of node: b8abced9-eb33-4313-b6c6-79e3b4f4f1b3 Starting introspection of node: 712ec08c-67d6-4d42-baa2-39d883b226c6 Starting introspection of node: 769786b0-fef1-4859-b784-0ca25969b75a Starting introspection of node: 804d63e8-d99e-40cf-8ea2-872df80ad28a Starting introspection of node: 20a2fe3a-630d-41c4-bba5-e24bb22a5fb0 Starting introspection of node: 790f1f0f-6efe-4c08-b710-03ce18ae912d Starting introspection of node: ef625302-86f5-4af4-ad38-6414d4b268fc Starting introspection of node: 009d0e77-c355-4904-952d-35f068725c20 Starting introspection of node: a7bf099a-208f-4746-a19a-631cccaee46f Starting introspection of node: fe4256fe-398b-46c6-964f-a0e1bd06fbd3 Starting introspection of node: 7ac0d4ec-a4a8-438b-8b2f-5f817a1a70cd Starting introspection of node: 1554cfe9-28c5-405f-9a28-7cbc71c8053f Starting introspection of node: 4795e1b0-0708-4be2-99de-412bd5401065 Starting introspection of node: 429aacc5-3476-40ff-86fd-9c7858c1b1d0 Starting introspection of node: 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa Starting introspection of node: 76e0f972-2a8b-4a74-916b-edcdca9b28d7 Starting introspection of node: 98140410-0311-407a-88c9-ce6d3bb9c32f Starting introspection of node: 8885598e-b9b8-4a16-9a19-a1dee15ddd44 Starting introspection of node: 0e26c0eb-92fa-4ab8-a13f-449653a6e19b Starting introspection of node: 1f0fd18e-2de1-4d12-8fd3-117c6b1968d1 Starting introspection of node: 560c181d-8da6-4997-9f5a-d899a6860cf5 Starting introspection of node: 372f7b66-9cf8-44c4-9136-b6554ed60e29 Starting introspection of node: 217323a6-614a-47db-90d0-7f00069d7038 Starting introspection of node: 71c8b4c6-7843-414a-8d3d-3f94f0d45089 Starting introspection of node: a736c564-a8bc-451f-996e-dd709551f45a Starting introspection of node: a0ef8d26-d61e-44cc-a058-764192663500 Starting introspection of node: 8dadaec4-2bc6-4f88-b205-5cdb016d0365 Starting introspection of node: 85fa62b6-10cd-4249-aaa3-cabfa3e85751 Starting introspection of node: eb9a4e05-6fc9-434b-9f33-6a31208dd888 Starting introspection of node: 4b819e73-4b3c-4746-a360-d4e2d6cb2c97 Starting introspection of node: 82e7ecfa-7a73-40e9-8ae3-55ba3b467ee8 Starting introspection of node: 391e5097-8cd7-40ac-8f2d-9f80adaf76ec Starting introspection of node: 29773e1b-05d6-4f2b-98e1-981f84b217ec Starting introspection of node: 40b034bc-0c99-4881-be8f-4e4ff34bce69 Starting introspection of node: 9658eb40-69be-4d79-b10d-15eee12bfd7a Starting introspection of node: 30b7480a-ff06-4510-8282-6d7c37073a49 Starting introspection of node: 216632a7-5012-4eb8-8117-2a4ac5245673 Starting introspection of node: 523d8a18-5f10-4c3e-82ff-c4ada7e06dba Starting introspection of node: aa8bc63c-0a3d-4924-9cc6-a81813126e07 Starting introspection of node: c274afdb-efc7-423b-abbd-2bb836ff1f1c Starting introspection of node: 9452b455-e67e-4fc8-b81b-0bc1ee7c97e8 Starting introspection of node: 503ae852-1dff-4bec-8ae6-0f9bcbd0d322 Starting introspection of node: 9d4be85a-40eb-41c1-a008-5922e39e865d Starting introspection of node: be7506a3-3211-4606-b1dd-20d822dfb4c0 Starting introspection of node: 011f6b22-059c-42c5-9e84-15b3c074d455 Starting introspection of node: cfb3ea0d-2f49-4bd0-aa7b-ab72a46438c0 Starting introspection of node: ff28c010-2d5b-4cc7-b111-270908a18e1f Starting introspection of node: 1da69901-7a14-4393-a61c-be02347039ec Starting introspection of node: 90b14609-c8ef-45bb-b173-0af540e3183f Starting introspection of node: 18c6a449-68f3-4b2d-93f5-cdfaa12ceeeb Starting introspection of node: ebbd548f-091c-410a-9e8f-7e783e947734 Starting introspection of node: 64dfd30c-50ec-41f1-9aa8-498ac3b8026a Starting introspection of node: 3c2abe43-42c4-4689-b0df-3d02d116bbe1 Waiting for discovery to finish... Discovery for UUID c7fdba2d-8177-4729-bdd7-5b2aedb58044 finished successfully. Discovery for UUID 73a75b9f-f4f1-4dd0-af62-9237cc2779d6 finished successfully. Discovery for UUID 389f6cab-f712-4a9a-86f3-c3c7e1d14076 finished successfully. Discovery for UUID 26554c4a-8318-4371-a11d-a18c5f7f9bf7 finished successfully. Discovery for UUID f50d37a6-ad5f-418f-8f9c-a2e480f2d442 finished successfully. Discovery for UUID 42105127-7526-4cd6-a5dd-ca41a4268e6a finished successfully. Discovery for UUID d259b087-0bab-455f-86ee-036fb1f23e5f finished successfully. Discovery for UUID 9fc5350f-df33-4b1b-bfbf-735c11f202ab finished successfully. Discovery for UUID 3d057044-1939-4ef8-ae61-c985bbb919f7 finished successfully. Discovery for UUID b8abced9-eb33-4313-b6c6-79e3b4f4f1b3 finished successfully. Discovery for UUID 769786b0-fef1-4859-b784-0ca25969b75a finished successfully. Discovery for UUID 20a2fe3a-630d-41c4-bba5-e24bb22a5fb0 finished successfully. Discovery for UUID ef625302-86f5-4af4-ad38-6414d4b268fc finished successfully. Discovery for UUID a7bf099a-208f-4746-a19a-631cccaee46f finished successfully. Discovery for UUID 7ac0d4ec-a4a8-438b-8b2f-5f817a1a70cd finished successfully. Discovery for UUID 4795e1b0-0708-4be2-99de-412bd5401065 finished with error: Introspection timeout Discovery for UUID 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa finished with error: Introspection timeout Discovery for UUID 98140410-0311-407a-88c9-ce6d3bb9c32f finished with error: Introspection timeout Discovery for UUID 0e26c0eb-92fa-4ab8-a13f-449653a6e19b finished with error: Introspection timeout Discovery for UUID 560c181d-8da6-4997-9f5a-d899a6860cf5 finished with error: Introspection timeout Discovery for UUID 217323a6-614a-47db-90d0-7f00069d7038 finished with error: Introspection timeout Discovery for UUID a736c564-a8bc-451f-996e-dd709551f45a finished successfully. Discovery for UUID 8dadaec4-2bc6-4f88-b205-5cdb016d0365 finished successfully. Discovery for UUID eb9a4e05-6fc9-434b-9f33-6a31208dd888 finished successfully. Discovery for UUID 82e7ecfa-7a73-40e9-8ae3-55ba3b467ee8 finished successfully. Discovery for UUID 29773e1b-05d6-4f2b-98e1-981f84b217ec finished successfully. Discovery for UUID 9658eb40-69be-4d79-b10d-15eee12bfd7a finished successfully. Discovery for UUID 216632a7-5012-4eb8-8117-2a4ac5245673 finished successfully. Discovery for UUID aa8bc63c-0a3d-4924-9cc6-a81813126e07 finished successfully. Discovery for UUID 9452b455-e67e-4fc8-b81b-0bc1ee7c97e8 finished successfully. Discovery for UUID 9d4be85a-40eb-41c1-a008-5922e39e865d finished successfully. Discovery for UUID 011f6b22-059c-42c5-9e84-15b3c074d455 finished successfully. Discovery for UUID ff28c010-2d5b-4cc7-b111-270908a18e1f finished successfully. Discovery for UUID 8e3eeef0-5bf8-48fa-86ae-89706bcc3cd1 finished successfully. Discovery for UUID 98ae6b68-cb01-4bf2-b8df-8ce1dc0b2de5 finished successfully. Discovery for UUID 9ed44703-8f39-4057-ace1-950415cde3eb finished successfully. Discovery for UUID 4e866cef-fb3c-4935-8f30-d0993e2a139c finished successfully. Discovery for UUID daf9009b-1a64-41ab-9a7f-a302407efdb9 finished successfully. Discovery for UUID 804d63e8-d99e-40cf-8ea2-872df80ad28a finished successfully. Discovery for UUID 009d0e77-c355-4904-952d-35f068725c20 finished successfully. Discovery for UUID 1554cfe9-28c5-405f-9a28-7cbc71c8053f finished with error: Introspection timeout Discovery for UUID 76e0f972-2a8b-4a74-916b-edcdca9b28d7 finished with error: Introspection timeout Discovery for UUID 1f0fd18e-2de1-4d12-8fd3-117c6b1968d1 finished with error: Introspection timeout Discovery for UUID 71c8b4c6-7843-414a-8d3d-3f94f0d45089 finished with error: Introspection timeout Discovery for UUID 85fa62b6-10cd-4249-aaa3-cabfa3e85751 finished successfully. Discovery for UUID 391e5097-8cd7-40ac-8f2d-9f80adaf76ec finished successfully. Discovery for UUID 30b7480a-ff06-4510-8282-6d7c37073a49 finished successfully. Discovery for UUID c274afdb-efc7-423b-abbd-2bb836ff1f1c finished successfully. Discovery for UUID be7506a3-3211-4606-b1dd-20d822dfb4c0 finished successfully. Discovery for UUID c7526e61-fe94-4857-bb39-33fb604de908 finished successfully. Discovery for UUID ed0b0f12-710c-4c3d-96b1-fcc1a5fc36a8 finished successfully. Discovery for UUID 712ec08c-67d6-4d42-baa2-39d883b226c6 finished successfully. Discovery for UUID fe4256fe-398b-46c6-964f-a0e1bd06fbd3 finished successfully. Discovery for UUID 8885598e-b9b8-4a16-9a19-a1dee15ddd44 finished with error: Introspection timeout Discovery for UUID a0ef8d26-d61e-44cc-a058-764192663500 finished successfully. Discovery for UUID 40b034bc-0c99-4881-be8f-4e4ff34bce69 finished successfully. Discovery for UUID 503ae852-1dff-4bec-8ae6-0f9bcbd0d322 finished successfully. Discovery for UUID beec77e1-bbd5-4c13-b5cf-5014779a3f6a finished successfully. Discovery for UUID 790f1f0f-6efe-4c08-b710-03ce18ae912d finished successfully. Discovery for UUID 372f7b66-9cf8-44c4-9136-b6554ed60e29 finished with error: Introspection timeout Discovery for UUID 523d8a18-5f10-4c3e-82ff-c4ada7e06dba finished successfully. Discovery for UUID 3859a0eb-4b10-483f-978b-d73f3b8c162a finished successfully. Discovery for UUID 4b819e73-4b3c-4746-a360-d4e2d6cb2c97 finished successfully. Discovery for UUID 429aacc5-3476-40ff-86fd-9c7858c1b1d0 finished with error: Introspection timeout Discovery for UUID cfb3ea0d-2f49-4bd0-aa7b-ab72a46438c0 finished successfully. Discovery for UUID 1da69901-7a14-4393-a61c-be02347039ec finished successfully. Discovery for UUID 18c6a449-68f3-4b2d-93f5-cdfaa12ceeeb finished successfully. Discovery for UUID 64dfd30c-50ec-41f1-9aa8-498ac3b8026a finished successfully. Discovery for UUID 90b14609-c8ef-45bb-b173-0af540e3183f finished successfully. Discovery for UUID 3c2abe43-42c4-4689-b0df-3d02d116bbe1 finished successfully. Discovery for UUID ebbd548f-091c-410a-9e8f-7e783e947734 finished successfully. Setting manageable nodes to available... Node c7fdba2d-8177-4729-bdd7-5b2aedb58044 has been set to available. Node 8e3eeef0-5bf8-48fa-86ae-89706bcc3cd1 has been set to available. Node 73a75b9f-f4f1-4dd0-af62-9237cc2779d6 has been set to available. Node c7526e61-fe94-4857-bb39-33fb604de908 has been set to available. Node 389f6cab-f712-4a9a-86f3-c3c7e1d14076 has been set to available. Node 98ae6b68-cb01-4bf2-b8df-8ce1dc0b2de5 has been set to available. Node 26554c4a-8318-4371-a11d-a18c5f7f9bf7 has been set to available. Node beec77e1-bbd5-4c13-b5cf-5014779a3f6a has been set to available. Node f50d37a6-ad5f-418f-8f9c-a2e480f2d442 has been set to available. Node 9ed44703-8f39-4057-ace1-950415cde3eb has been set to available. Node 42105127-7526-4cd6-a5dd-ca41a4268e6a has been set to available. Node ed0b0f12-710c-4c3d-96b1-fcc1a5fc36a8 has been set to available. Node d259b087-0bab-455f-86ee-036fb1f23e5f has been set to available. Node 4e866cef-fb3c-4935-8f30-d0993e2a139c has been set to available. Node 9fc5350f-df33-4b1b-bfbf-735c11f202ab has been set to available. Node 3859a0eb-4b10-483f-978b-d73f3b8c162a has been set to available. Node 3d057044-1939-4ef8-ae61-c985bbb919f7 has been set to available. Node daf9009b-1a64-41ab-9a7f-a302407efdb9 has been set to available. Node b8abced9-eb33-4313-b6c6-79e3b4f4f1b3 has been set to available. Node 712ec08c-67d6-4d42-baa2-39d883b226c6 has been set to available. Node 769786b0-fef1-4859-b784-0ca25969b75a has been set to available. Node 804d63e8-d99e-40cf-8ea2-872df80ad28a has been set to available. Node 20a2fe3a-630d-41c4-bba5-e24bb22a5fb0 has been set to available. Node 790f1f0f-6efe-4c08-b710-03ce18ae912d has been set to available. Node ef625302-86f5-4af4-ad38-6414d4b268fc has been set to available. Node 009d0e77-c355-4904-952d-35f068725c20 has been set to available. Node a7bf099a-208f-4746-a19a-631cccaee46f has been set to available. Node fe4256fe-398b-46c6-964f-a0e1bd06fbd3 has been set to available. Node 7ac0d4ec-a4a8-438b-8b2f-5f817a1a70cd has been set to available. Node 1554cfe9-28c5-405f-9a28-7cbc71c8053f has been set to available. Node 4795e1b0-0708-4be2-99de-412bd5401065 has been set to available. Node 429aacc5-3476-40ff-86fd-9c7858c1b1d0 has been set to available. Node 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa has been set to available. Node 76e0f972-2a8b-4a74-916b-edcdca9b28d7 has been set to available. Node 98140410-0311-407a-88c9-ce6d3bb9c32f has been set to available. Node 8885598e-b9b8-4a16-9a19-a1dee15ddd44 has been set to available. Node 0e26c0eb-92fa-4ab8-a13f-449653a6e19b has been set to available. Node 1f0fd18e-2de1-4d12-8fd3-117c6b1968d1 has been set to available. Node 560c181d-8da6-4997-9f5a-d899a6860cf5 has been set to available. Node 372f7b66-9cf8-44c4-9136-b6554ed60e29 has been set to available. Node 217323a6-614a-47db-90d0-7f00069d7038 has been set to available. Node 71c8b4c6-7843-414a-8d3d-3f94f0d45089 has been set to available. Node a736c564-a8bc-451f-996e-dd709551f45a has been set to available. Node a0ef8d26-d61e-44cc-a058-764192663500 has been set to available. Node 8dadaec4-2bc6-4f88-b205-5cdb016d0365 has been set to available. Node 85fa62b6-10cd-4249-aaa3-cabfa3e85751 has been set to available. Node eb9a4e05-6fc9-434b-9f33-6a31208dd888 has been set to available. Node 4b819e73-4b3c-4746-a360-d4e2d6cb2c97 has been set to available. Node 82e7ecfa-7a73-40e9-8ae3-55ba3b467ee8 has been set to available. Node 391e5097-8cd7-40ac-8f2d-9f80adaf76ec has been set to available. Node 29773e1b-05d6-4f2b-98e1-981f84b217ec has been set to available. Node 40b034bc-0c99-4881-be8f-4e4ff34bce69 has been set to available. Node 9658eb40-69be-4d79-b10d-15eee12bfd7a has been set to available. Node 30b7480a-ff06-4510-8282-6d7c37073a49 has been set to available. Node 216632a7-5012-4eb8-8117-2a4ac5245673 has been set to available. Node 523d8a18-5f10-4c3e-82ff-c4ada7e06dba has been set to available. Node aa8bc63c-0a3d-4924-9cc6-a81813126e07 has been set to available. Node c274afdb-efc7-423b-abbd-2bb836ff1f1c has been set to available. Node 9452b455-e67e-4fc8-b81b-0bc1ee7c97e8 has been set to available. Node 503ae852-1dff-4bec-8ae6-0f9bcbd0d322 has been set to available. Node 9d4be85a-40eb-41c1-a008-5922e39e865d has been set to available. Node be7506a3-3211-4606-b1dd-20d822dfb4c0 has been set to available. Node 011f6b22-059c-42c5-9e84-15b3c074d455 has been set to available. Node cfb3ea0d-2f49-4bd0-aa7b-ab72a46438c0 has been set to available. Node ff28c010-2d5b-4cc7-b111-270908a18e1f has been set to available. Node 1da69901-7a14-4393-a61c-be02347039ec has been set to available. Node 90b14609-c8ef-45bb-b173-0af540e3183f has been set to available. Node 18c6a449-68f3-4b2d-93f5-cdfaa12ceeeb has been set to available. Node ebbd548f-091c-410a-9e8f-7e783e947734 has been set to available. Node 64dfd30c-50ec-41f1-9aa8-498ac3b8026a has been set to available. Node 3c2abe43-42c4-4689-b0df-3d02d116bbe1 has been set to available. Discovery completed. [stack@dell-undercloud ~]$ ironic node-show 1554cfe9-28c5-405f-9a28-7cbc71c8053f +------------------------+-----------------------------------------------------------------------+ | Property | Value | +------------------------+-----------------------------------------------------------------------+ | target_power_state | None | | extra | {u'on_discovery': u'true'} | | last_error | None | | updated_at | 2015-09-16T21:45:27+00:00 | | maintenance_reason | None | | provision_state | available | | uuid | 1554cfe9-28c5-405f-9a28-7cbc71c8053f | | console_enabled | False | | target_provision_state | None | | maintenance | True | | inspection_started_at | None | | inspection_finished_at | None | | power_state | power on | | driver | pxe_ipmitool | | reservation | None | | properties | {u'memory_mb': u'16384', u'cpu_arch': u'x86_64', u'local_gb': u'500', | | | u'cpus': u'4', u'capabilities': u'boot_option:local'} | | instance_uuid | None | | name | None | | driver_info | {u'ipmi_password': u'******', u'ipmi_address': u'10.17.64.109', | | | u'ipmi_username': u'root', u'deploy_kernel': u'bf8c3775-bb1d-466e- | | | bee1-057472379440', u'deploy_ramdisk': u'8ea8b4b4-004a- | | | 491d-a900-f62b82080fdf'} | | created_at | 2015-09-16T19:31:09+00:00 | | driver_internal_info | {u'clean_steps': None} | | chassis_uuid | | | instance_info | {} | +------------------------+-----------------------------------------------------------------------+ [stack@dell-undercloud ~]$ ironic node-show 1554cfe9-28c5-405f-9a28-7cbc71c8053f +------------------------+-----------------------------------------------------------------------+ | Property | Value | +------------------------+-----------------------------------------------------------------------+ | target_power_state | None | | extra | {u'newly_discovered': u'true', u'block_devices': {u'serials': | | | [u'35H5XVPLS']}, u'hardware_swift_object': u'extra_hardware- | | | 1554cfe9-28c5-405f-9a28-7cbc71c8053f'} | | last_error | None | | updated_at | 2015-09-16T21:50:49+00:00 | | maintenance_reason | None | | provision_state | available | | uuid | 1554cfe9-28c5-405f-9a28-7cbc71c8053f | | console_enabled | False | | target_provision_state | None | | maintenance | True | | inspection_started_at | None | | inspection_finished_at | None | | power_state | power off | | driver | pxe_ipmitool | | reservation | None | | properties | {u'memory_mb': u'16384', u'cpu_arch': u'x86_64', u'local_gb': u'464', | | | u'cpus': u'8', u'capabilities': u'boot_option:local'} | | instance_uuid | None | | name | None | | driver_info | {u'ipmi_password': u'******', u'ipmi_address': u'10.17.64.109', | | | u'ipmi_username': u'root', u'deploy_kernel': u'bf8c3775-bb1d-466e- | | | bee1-057472379440', u'deploy_ramdisk': u'8ea8b4b4-004a- | | | 491d-a900-f62b82080fdf'} | | created_at | 2015-09-16T19:31:09+00:00 | | driver_internal_info | {u'clean_steps': None} | | chassis_uuid | | | instance_info | {} | +------------------------+-----------------------------------------------------------------------+ [stack@dell-undercloud ~]$ [stack@dell-undercloud ~]$ [stack@dell-undercloud ~]$ [stack@dell-undercloud ~]$ ironic node-show 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa +------------------------+-----------------------------------------------------------------------+ | Property | Value | +------------------------+-----------------------------------------------------------------------+ | target_power_state | None | | extra | {u'on_discovery': u'true'} | | last_error | None | | updated_at | 2015-09-16T21:47:24+00:00 | | maintenance_reason | None | | provision_state | available | | uuid | 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa | | console_enabled | False | | target_provision_state | None | | maintenance | True | | inspection_started_at | None | | inspection_finished_at | None | | power_state | power on | | driver | pxe_ipmitool | | reservation | None | | properties | {u'memory_mb': u'16384', u'cpu_arch': u'x86_64', u'local_gb': u'500', | | | u'cpus': u'4', u'capabilities': u'boot_option:local'} | | instance_uuid | None | | name | None | | driver_info | {u'ipmi_password': u'******', u'ipmi_address': u'10.17.64.100', | | | u'ipmi_username': u'root', u'deploy_kernel': u'bf8c3775-bb1d-466e- | | | bee1-057472379440', u'deploy_ramdisk': u'8ea8b4b4-004a- | | | 491d-a900-f62b82080fdf'} | | created_at | 2015-09-16T19:31:10+00:00 | | driver_internal_info | {u'clean_steps': None} | | chassis_uuid | | | instance_info | {} | +------------------------+-----------------------------------------------------------------------+ [stack@dell-undercloud ~]$ ironic node-show 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa +------------------------+-----------------------------------------------------------------------+ | Property | Value | +------------------------+-----------------------------------------------------------------------+ | target_power_state | None | | extra | {u'newly_discovered': u'true', u'block_devices': {u'serials': | | | [u'35H1625LS']}, u'hardware_swift_object': u'extra_hardware-5d32d61f- | | | 0e62-4f38-b11a-5fdd7e9265fa'} | | last_error | None | | updated_at | 2015-09-16T21:57:09+00:00 | | maintenance_reason | None | | provision_state | available | | uuid | 5d32d61f-0e62-4f38-b11a-5fdd7e9265fa | | console_enabled | False | | target_provision_state | None | | maintenance | True | | inspection_started_at | None | | inspection_finished_at | None | | power_state | power off | | driver | pxe_ipmitool | | reservation | None | | properties | {u'memory_mb': u'16384', u'cpu_arch': u'x86_64', u'local_gb': u'464', | | | u'cpus': u'8', u'capabilities': u'boot_option:local'} | | instance_uuid | None | | name | None | | driver_info | {u'ipmi_password': u'******', u'ipmi_address': u'10.17.64.100', | | | u'ipmi_username': u'root', u'deploy_kernel': u'bf8c3775-bb1d-466e- | | | bee1-057472379440', u'deploy_ramdisk': u'8ea8b4b4-004a- | | | 491d-a900-f62b82080fdf'} | | created_at | 2015-09-16T19:31:10+00:00 | | driver_internal_info | {u'clean_steps': None} | | chassis_uuid | | | instance_info | {} | +------------------------+-----------------------------------------------------------------------+ Version-Release number of selected component (if applicable): 7.0 How reproducible: Steps to Reproduce: 1. import 72 nodes into openstack baremetal 2. openstack baremetal configure boot 3. openstack baremetal introspection bulk start after a few hours, some nodes will fail introspection with timeout message Actual results: Expected results: should be able to inspect all nodes Additional info: