Bug 1225226 - Nodes discovery fails when instackenv.json file contains nodes mac addresses.
Summary: Nodes discovery fails when instackenv.json file contains nodes mac addresses.
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
Target Milestone: ga
: Director
Assignee: Dmitry Tantsur
QA Contact: Marius Cornea
Depends On:
TreeView+ depends on / blocked
Reported: 2015-05-26 21:20 UTC by Marius Cornea
Modified: 2016-04-18 15:13 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2015-08-05 13:51:54 UTC
Target Upstream Version:

Attachments (Terms of Use)
ironic logs (13.38 KB, text/plain)
2015-05-26 21:20 UTC, Marius Cornea
no flags Details
more ironic logs (88.74 KB, text/plain)
2015-06-10 14:39 UTC, Ofer Blaut
no flags Details

System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1308714 None None None Never
Red Hat Product Errata RHEA-2015:1549 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform director Release 2015-08-05 17:49:10 UTC

Internal Links: 1308714

Description Marius Cornea 2015-05-26 21:20:15 UTC
Created attachment 1030183 [details]
ironic logs

Description of problem:
Nodes discovery fails when instackenv.json file contains the nodes mac addresses.

Version-Release number of selected component (if applicable):

How reproducible:
Poodle installation.

Steps to Reproduce:
1. Deploy undercloud
2. instack-ironic-deployment --nodes-json instackenv.json --register-nodes
3. instack-ironic-deployment --discover-nodes

Actual results:

[stack@bldr16cc09 ~]$ instack-ironic-deployment --discover-nodes
Preparing for deployment...
  Discovering nodes.
    Sending node ID bda62f11-c51d-4dc2-9678-8568b0df2b52 to discoverd for discovery ... DONE.
    Sending node ID 320892bc-8a4b-466d-82e4-46e6cda9be4d to discoverd for discovery ... DONE.
    Sending node ID 1ef01a1a-e985-4c42-8a2a-fa9493927595 to discoverd for discovery ... DONE.
   Polling discoverd for discovery results ... 
   Result for node bda62f11-c51d-4dc2-9678-8568b0df2b52 is ... DISCOVERED.
Node bda62f11-c51d-4dc2-9678-8568b0df2b52 is locked by host localhost.localdomain, please retry after the current operation is completed. (HTTP 409)

Expected results:
Nodes hardware parameters get discovered.

Additional info:
It looks that if I remove the mac addresses from the instackenv.json file the 'port with mac address already exits' error message disappears but discovery still fails for some of the nodes. I'm attaching relevant logs and steps I did.

Comment 3 Dmitry Tantsur 2015-05-27 07:57:07 UTC
It's definitely not a discoverd bug. Either my retrying solution in ironicclient is incomplete, or we should tune a couple of things in instack-undercloud. I'll have a look.

Comment 4 Dmitry Tantsur 2015-05-27 09:34:24 UTC
Something like https://review.gerrithub.io/#/c/234406/ should help

Comment 5 Marius Cornea 2015-05-27 10:11:10 UTC
Tested the patch and nodes got discovered with provision state set to available this time. Thanks! 

One thing I noticed is that errors still show up in the openstack-ironic-discoverd log, maybe showing those with warning severity would be better:

ERROR:ironicclient.common.http:Error contacting Ironic server: A port with MAC address d8:d3:85:be:01:0c already exists. (HTTP 409). Attempt 6 of 6
ERROR:ironicclient.common.http:Error contacting Ironic server: Node 5394855d-0423-40a4-b86a-45ebfd09abe1 can not be updated while a state transition is in progress. (HTTP 409). Attempt 6 of 6

Comment 8 Ofer Blaut 2015-06-10 14:39:18 UTC
Created attachment 1037336 [details]
more ironic logs

Comment 10 Alexander Chuzhoy 2015-06-10 17:02:58 UTC
If I remove the MAC addresses from the instackenv.json file prior to registering the nodes, then only one node gets registered upon running "instack-ironic-deployment --nodes-json instackenv.json --register-nodes".

I actually had to re-run the above command 4 times to see 4 nodes in the output from "ironic node-list", where one node was added after every run of the command to the list.

Comment 12 errata-xmlrpc 2015-08-05 13:51:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.