Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RDO tickets are now tracked in Jira https://issues.redhat.com/projects/RDO/issues/

Bug 1209176

Summary: openstack Refusing to introspect node with provision state "available" and maintenance mode off
Product: [Community] RDO Reporter: Ola Pavlenko <opavlenk>
Component: openstack-ironic-discoverdAssignee: Dmitry Tantsur <dtantsur>
Status: CLOSED NOTABUG QA Contact: Toure Dunnon <tdunnon>
Severity: high Docs Contact:
Priority: unspecified    
Version: trunkCC: dtantsur, opavlenk
Target Milestone: ---   
Target Release: Kilo   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-04-07 12:45:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
json file was used to import the nodes
none
journalctl none

Description Ola Pavlenko 2015-04-06 15:06:31 UTC
Description of problem:
running openstack baremetal introspection start all and got ERROR: openstack Refusing to introspect node 904e28ad-40a0-45b9-befc-21388f615bc8 with provision state "available" and maintenance mode off

Version-Release number of selected component (if applicable):
$ rpm -qa | grep rdo
python-rdomanager-oscplugin-0.0.1-6ede03c.el7.centos.noarch
rdo-release-kilo-0.noarch


How reproducible:
100%

Steps to Reproduce:
1.import nodes using default instackenv.json file (VMs)
2. run openstack baremetal introspection all start
3.

Actual results:
]$ openstack baremetal introspection all start
/usr/lib/python2.7/site-packages/novaclient/v1_1/__init__.py:30: UserWarning: Module novaclient.v1_1 is deprecated (taken as a basis for novaclient.v2). The preferable way to get client class or object you can find in novaclient.client module.
  warnings.warn("Module novaclient.v1_1 is deprecated (taken as a basis for "
ERROR: openstack Refusing to introspect node 904e28ad-40a0-45b9-befc-21388f615bc8 with provision state "available" and maintenance mode off

Expected results:
discovery was successfully completed 

Additional info:
$ ironic node-list
+--------------------------------------+---------------+-------------+--------------------+-------------+
| UUID                                 | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+---------------+-------------+--------------------+-------------+
| 904e28ad-40a0-45b9-befc-21388f615bc8 | None          | power off   | None               | False       |
| f40a3388-c337-4083-bb5b-ca7b7a877728 | None          | power off   | None               | False       |
+--------------------------------------+---------------+-------------+--------------------+-------------+

Comment 1 Ola Pavlenko 2015-04-06 15:11:50 UTC
Created attachment 1011399 [details]
json file was used to import the nodes

Comment 2 Ola Pavlenko 2015-04-06 15:39:34 UTC
Probably related to the same issue:
running openstack baremetal introspection all status returns:

/usr/lib/python2.7/site-packages/novaclient/v1_1/__init__.py:30: UserWarning: Module novaclient.v1_1 is deprecated (taken as a basis for novaclient.v2). The preferable way to get client class or object you can find in novaclient.client module.
  warnings.warn("Module novaclient.v1_1 is deprecated (taken as a basis for "
ERROR: openstack Could not find node 904e28ad-40a0-45b9-befc-21388f615bc8 in cache


when:
$ ironic node-list
+--------------------------------------+---------------+-------------+--------------------+-------------+
| UUID                                 | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+---------------+-------------+--------------------+-------------+
| 904e28ad-40a0-45b9-befc-21388f615bc8 | None          | power off   | None               | False       |
| f40a3388-c337-4083-bb5b-ca7b7a877728 | None          | power off   | None               | False       |
+--------------------------------------+---------------+-------------+--------------------+-------------+

Comment 3 Dmitry Tantsur 2015-04-07 11:21:46 UTC
Hi!

Please provide your version of python-ironicclient. We had a premature rebase recently that might cause this.

Comment 4 Ola Pavlenko 2015-04-07 11:36:05 UTC
(In reply to Dmitry Tantsur from comment #3)
> Hi!
> 
> Please provide your version of python-ironicclient. We had a premature
> rebase recently that might cause this.

Hi Dmitry,

Here it is:
$ rpm -qa | grep python-ironicclient
python-ironicclient-0.4.1.25-g3b171c5.el7.centos.noarch

Comment 5 Dmitry Tantsur 2015-04-07 11:50:41 UTC
That's weird, I see that `ironic node-list` returns state None (as expected for now) and you have a correct version of ironicclient...

One more guess: could it happen that you updated ironicclient recently but didn't restart either of openstack-ironic-discoverd, openstack-ironic-conductor or openstack-ironic-api? In other words, does the problem persists if you restart all 3 and retry?

Also please provide your versions of openstack-ironic-discoverd and openstack-ironic-api just in case. 

Finally, please provide output of $ sudo journalctl -u openstack-ironic-api

Thanks!

Comment 6 Ola Pavlenko 2015-04-07 12:33:26 UTC
(In reply to Dmitry Tantsur from comment #5)
> That's weird, I see that `ironic node-list` returns state None (as expected
> for now) and you have a correct version of ironicclient...
> 
> One more guess: could it happen that you updated ironicclient recently but
> didn't restart either of openstack-ironic-discoverd,
> openstack-ironic-conductor or openstack-ironic-api? In other words, does the
> problem persists if you restart all 3 and retry?
> 
> Also please provide your versions of   and
> openstack-ironic-api just in case. 
> 
> Finally, please provide output of $ sudo journalctl -u openstack-ironic-api
> 
> Thanks!

Actually I did updated the system a couple of days ago, but didn't restart anything.
Is restart required after yum update?

$ rpm -qa | grep openstack-ironic
openstack-ironic-common-2015.1-dev546.gf102adf.el7.centos.noarch
openstack-ironic-discoverd-1.1.0-0.99.20150327.1456git.el7.centos.noarch
openstack-ironic-api-2015.1-dev546.gf102adf.el7.centos.noarch
openstack-ironic-conductor-2015.1-dev546.gf102adf.el7.centos.noarch

Comment 7 Dmitry Tantsur 2015-04-07 12:34:34 UTC
Actually it should work with just yum update, but sometimes doesn't work for me for some reason, so it's better to double-check before we dive in :)

Comment 8 Ola Pavlenko 2015-04-07 12:35:25 UTC
Created attachment 1011743 [details]
journalctl

the output of " sudo journalctl -u openstack-ironic-api" to journalctl_ironic.log after restart

*** ironic-discoverd, conductor and api were  restarted

Comment 9 Dmitry Tantsur 2015-04-07 12:38:01 UTC
Great, now please retry discovery and collect the logs again if the problem persists.

Comment 10 Ola Pavlenko 2015-04-07 12:45:03 UTC
(In reply to Dmitry Tantsur from comment #9)
> Great, now please retry discovery and collect the logs again if the problem
> persists.

restarted the ironic-discoverd, conductor and api , now seems to be working. ran the commands again and didn't get any error
Closing as "NOT A BUG"
Thanks