Successful introspection notes are confusing. Environment: openstack-ironic-common-10.0.1-0.20180123173432.2c630d9.el7ost.noarch python-ironic-inspector-client-3.0.0-0.20171206024751.e4ceea7.el7ost.noarch openstack-ironic-conductor-10.0.1-0.20180123173432.2c630d9.el7ost.noarch puppet-ironic-12.2.0-0.20180119235614.dd2b63c.el7ost.noarch python-ironic-lib-2.11.0-0.20171026141645.effbee9.el7ost.noarch openstack-ironic-staging-drivers-0.8.1-0.20180118014800.ae60c0b.el7ost.noarch python2-ironicclient-2.1.0-0.20180112022138.5a427ee.el7ost.noarch openstack-ironic-api-10.0.1-0.20180123173432.2c630d9.el7ost.noarch openstack-ironic-inspector-7.0.1-0.20180122225049.0cf3316.el7ost.noarch instack-undercloud-8.1.1-0.20180117134321.el7ost.noarch I tested the following introspection command with 2 scenarios: "openstack overcloud node introspect --all-manageable --provide" 1) All the nodes in ironic were set to available. In this case the introspection bypassed all nodes and printed the following messages: Successfully introspected nodes. Nodes introspected successfully. Introspection completed. ######################################################################## 2) All the nodes in ironic were set to managed. In this case the introspection actually occured and the following was printed. Introspection of node 9d0ac4c0-f0de-415d-a00c-48d55a73715f completed. Status:SUCCESS. Errors:None Introspection of node 75c5e888-4dd8-4d77-9499-0c48d544e0fd completed. Status:SUCCESS. Errors:None Introspection of node ba06d395-4233-4e4f-ab0d-d0b5fadfb1d2 completed. Status:SUCCESS. Errors:None Introspection of node 3e20b762-6e1b-4527-870e-407b12d0cc82 completed. Status:SUCCESS. Errors:None Introspection of node fcf5871d-9ba0-40c1-8d6d-4923228f715c completed. Status:SUCCESS. Errors:None Introspection of node 339c56ec-513b-4e26-8a2c-699fc561be2d completed. Status:SUCCESS. Errors:None Introspection of node c1480e8b-9ffe-4a6a-a5b1-1bd5c92f9c8e completed. Status:SUCCESS. Errors:None Introspection of node a5d410d0-c947-4048-961d-59816649c7a6 completed. Status:SUCCESS. Errors:None Successfully introspected nodes. Nodes introspected successfully. Introspection completed. ################################################################33 Issues: 1. One of the following lines is redundant: A. Successfully introspected nodes. B. Nodes introspected successfully. 2. Need to print a respective message when the introspection didn't occur on any node.
Environment: openstack-tripleo-common-8.6.1-6.el7ost.noarch Success note: Waiting for introspection to finish... Started Mistral Workflow tripleo.baremetal.v1.introspect_manageable_nodes. Execution ID: c6d74afe-2fe1-411d-b942-4b0d97017af4 Waiting for messages on queue 'tripleo' with no timeout. Introspection of node 08606c14-eac6-4e29-a9e0-e8c3c4925679 completed. Status:SUCCESS. Errors:None Introspection of node 6b5514db-ef09-4366-9131-89587749b4a2 completed. Status:SUCCESS. Errors:None Successfully introspected 2 node(s). Introspection completed. Started Mistral Workflow tripleo.baremetal.v1.provide_manageable_nodes. Execution ID: ad9433e7-dc45-4776-988a-d930a6920ab2 Waiting for messages on queue 'tripleo' with no timeout. 2 node(s) successfully moved to the "available" state.
In following along with how bug is reported. In first scenarios 1 and 2 we used this command. openstack overcloud node introspect --all-manageable --provide 1) This case all nodes are available state when running cmd: "openstack overcloud node introspect --all-manageable --provide" (undercloud) [stack@undercloud-0 ~]$ ironic node-list The "ironic" CLI is deprecated and will be removed in the S* release. Please use the "openstack baremetal" CLI instead. +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | 7fe9ae08-637a-4724-b661-206729b5e390 | ceph-0 | None | power off | available | False | | aeb9ba5b-444c-4101-8e5a-328d3c1282ad | ceph-1 | None | power off | available | False | | aebf4426-c26e-428f-8787-975448669425 | ceph-2 | None | power off | available | False | | b41c83bc-250f-44c5-81cf-6e1ea2724cc0 | compute-0 | None | power off | available | False | | 2598d417-1d23-47ff-afef-65bed0852a31 | controller-0 | None | power off | available | False | | 7b2c73c6-9434-4f0f-84e4-20b6f1ff5796 | controller-1 | None | power off | available | False | | e4d7371f-c29a-4e28-a617-4959ffa0a7f5 | controller-2 | None | power off | available | False | | ad3b9b43-ab4a-46e0-97e3-9ae7fddf4a23 | ironic-0 | None | power off | available | False | | ce8b45a8-4c69-4c70-a3b6-b2a6e9fbad4c | ironic-1 | None | power off | available | False | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ (undercloud) [stack@undercloud-0 ~]$ (undercloud) [stack@undercloud-0 ~]$ (undercloud) [stack@undercloud-0 ~]$ (undercloud) [stack@undercloud-0 ~]$ (undercloud) [stack@undercloud-0 ~]$ openstack overcloud node introspect --all-manageable --provide Waiting for introspection to finish... Started Mistral Workflow tripleo.baremetal.v1.introspect_manageable_nodes. Execution ID: 18e582f3-4c66-4973-a066-efaa07dde89f Waiting for messages on queue 'tripleo' with no timeout. No manageable nodes to introspect. Check node states and maintenance. Exception introspecting nodes: No manageable nodes to introspect. Check node states and maintenance. 2) This case all nodes are manageable when running cmd: "openstack overcloud node introspect --all-manageable --provide" (undercloud) [stack@undercloud-0 ~]$ ironic node-list The "ironic" CLI is deprecated and will be removed in the S* release. Please use the "openstack baremetal" CLI instead. +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | 7fe9ae08-637a-4724-b661-206729b5e390 | ceph-0 | None | power off | manageable | False | | aeb9ba5b-444c-4101-8e5a-328d3c1282ad | ceph-1 | None | power off | manageable | False | | aebf4426-c26e-428f-8787-975448669425 | ceph-2 | None | power off | manageable | False | | b41c83bc-250f-44c5-81cf-6e1ea2724cc0 | compute-0 | None | power off | manageable | False | | 2598d417-1d23-47ff-afef-65bed0852a31 | controller-0 | None | power off | manageable | False | | 7b2c73c6-9434-4f0f-84e4-20b6f1ff5796 | controller-1 | None | power off | manageable | False | | e4d7371f-c29a-4e28-a617-4959ffa0a7f5 | controller-2 | None | power off | manageable | False | | ad3b9b43-ab4a-46e0-97e3-9ae7fddf4a23 | ironic-0 | None | power off | manageable | False | | ce8b45a8-4c69-4c70-a3b6-b2a6e9fbad4c | ironic-1 | None | power off | manageable | False | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ (undercloud) [stack@undercloud-0 ~]$ openstack overcloud node introspect --all-manageable --provide Waiting for introspection to finish... Started Mistral Workflow tripleo.baremetal.v1.introspect_manageable_nodes. Execution ID: 51e0bcea-46f2-45cb-8992-a5db53c1f87c Waiting for messages on queue 'tripleo' with no timeout. ^[[OIntrospection of node 7fe9ae08-637a-4724-b661-206729b5e390 completed. Status:SUCCESS. Errors:None Introspection of node aebf4426-c26e-428f-8787-975448669425 completed. Status:SUCCESS. Errors:None Introspection of node aeb9ba5b-444c-4101-8e5a-328d3c1282ad completed. Status:SUCCESS. Errors:None Introspection of node b41c83bc-250f-44c5-81cf-6e1ea2724cc0 completed. Status:SUCCESS. Errors:None Introspection of node ad3b9b43-ab4a-46e0-97e3-9ae7fddf4a23 completed. Status:SUCCESS. Errors:None Introspection of node ce8b45a8-4c69-4c70-a3b6-b2a6e9fbad4c completed. Status:SUCCESS. Errors:None Introspection of node 2598d417-1d23-47ff-afef-65bed0852a31 completed. Status:SUCCESS. Errors:None Introspection of node 7b2c73c6-9434-4f0f-84e4-20b6f1ff5796 completed. Status:SUCCESS. Errors:None Introspection of node e4d7371f-c29a-4e28-a617-4959ffa0a7f5 completed. Status:SUCCESS. Errors:None Successfully introspected 9 node(s). Introspection completed. Started Mistral Workflow tripleo.baremetal.v1.provide_manageable_nodes. Execution ID: 9e811489-1f32-4224-b0de-52fb05bda5fe Waiting for messages on queue 'tripleo' with no timeout. 9 node(s) successfully moved to the "available" state. ======
FailedQA: One of the failed nodes print out a dump of information to screen which should get cleaned up when nodes won't introspect. (undercloud) [stack@undercloud-0 ~]$ ironic node-list The "ironic" CLI is deprecated and will be removed in the S* release. Please use the "openstack baremetal" CLI instead. +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | 84afdc8d-5ef1-40d6-9699-ceed2038eb51 | ceph-0 | None | power off | manageable | False | | 96c6189f-4cc7-4b74-b05a-ef9d4407bf7f | ceph-1 | None | power off | manageable | False | | ea2a88ea-5c3b-429f-9eca-f5a2b4369964 | ceph-2 | None | power off | manageable | False | | b8e38433-bdb2-4efa-8e9f-1c43f9a15404 | compute-0 | None | power off | manageable | False | | e8a8dd6c-be5d-421b-938a-019f87997059 | controller-0 | None | power off | manageable | False | | ca84b980-f81a-4a7e-a403-5826323ca778 | controller-1 | None | power off | manageable | False | | 91fe16ea-fb0f-4e56-bdbd-738d99ecb892 | controller-2 | None | power off | manageable | False | | 948f1601-7f8c-4874-8d44-ce26ae1bc8b8 | ironic-0 | None | power off | manageable | False | | e7b8855a-37c0-429c-9343-13f0e7926572 | ironic-1 | None | None | manageable | True | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ This is the output when ironic-1 node vbmc port is stopped. (undercloud) [stack@undercloud-0 ~]$ openstack overcloud node introspect --all-manageable --provide Waiting for introspection to finish... Started Mistral Workflow tripleo.baremetal.v1.introspect_manageable_nodes. Execution ID: a5005fbb-3927-4fc6-b506-a9f6f86dc6b5 Waiting for messages on queue 'tripleo' with no timeout. ^[[OIntrospection of node 84afdc8d-5ef1-40d6-9699-ceed2038eb51 completed. Status:SUCCESS. Errors:None Introspection of node 96c6189f-4cc7-4b74-b05a-ef9d4407bf7f completed. Status:SUCCESS. Errors:None Introspection of node ea2a88ea-5c3b-429f-9eca-f5a2b4369964 completed. Status:SUCCESS. Errors:None Introspection of node b8e38433-bdb2-4efa-8e9f-1c43f9a15404 completed. Status:SUCCESS. Errors:None Introspection of node e8a8dd6c-be5d-421b-938a-019f87997059 completed. Status:SUCCESS. Errors:None ^[[I^[[OIntrospection of node ca84b980-f81a-4a7e-a403-5826323ca778 completed. Status:SUCCESS. Errors:None Introspection of node 91fe16ea-fb0f-4e56-bdbd-738d99ecb892 completed. Status:SUCCESS. Errors:None Introspection of node 948f1601-7f8c-4874-8d44-ce26ae1bc8b8 completed. Status:SUCCESS. Errors:None Introspection of node e7b8855a-37c0-429c-9343-13f0e7926572 timed out. Retrying 1 nodes that failed introspection. Attempt 2 of 3 ^[[I^[[O^[[I^[[O^[[I^[[O^[[I^[[O^[[I^[[OIntrospection of node e7b8855a-37c0-429c-9343-13f0e7926572 timed out. Retrying 1 nodes that failed introspection. Attempt 3 of 3 ^[[I^[[O^[[I^[[O ^[[I^[[O^[[I^[[OIntrospection of node e7b8855a-37c0-429c-9343-13f0e7926572 timed out. Retry limit reached with 1 nodes still failing introspection {u'status': u'RUNNING', u'node_uuids': [u'e7b8855a-37c0-429c-9343-13f0e7926572'], u'failed_introspection': [u'e7b8855a-37c0-429c-9343-13f0e7926572'], u'result': None, u'introspected_nodes': {u'948f1601-7f8c-4874-8d44-ce26ae1bc8b8': {u'uuid': u'948f1601-7f8c-4874-8d44-ce26ae1bc8b8', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/948f1601-7f8c-4874-8d44-ce26ae1bc8b8', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:12:35', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2 018-05-11T20:07:15'}, u'b8e38433-bdb2-4efa-8e9f-1c43f9a15404': {u'uuid': u'b8e38433-bdb2-4efa-8e9f-1c43f9a15404', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspec Introspection of node e7b8855a-37c0-429c-9343-13f0e7926572 timed out. Retrying 1 nodes that failed introspection. Attempt 2 of 3 ^[[I^[[O^[[I^[[O^[[I^[[O^[[I^[[O^[[I^[[OIntrospection of node e7b8855a-37c0-429c-9343-13f0e7926572 timed out. Retrying 1 nodes that failed introspection. Attempt 3 of 3 018-05-11T20:07:15'}, u'b8e38433-bdb2-4efa-8e9f-1c43f9a15404': {u'uuid': u'b8e38433-bdb2-4efa-8e9f-1c43f9a15404', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspec tion/b8e38433-bdb2-4efa-8e9f-1c43f9a15404', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:17', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:12'}, u'e7b8855a-37c0-429c-9343-13f0e7926572': {u'uuid': u'e7b8855a-37c0-429c-9343-13f0e7926572', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/e7b8855a-37c0-429c-9343-13f0e7926572', u'rel': u'self'}], u'finished_at': None, u'state': u'waiting', u'finished': False, u'error': None, u'started_at': u'2018-05-11T20:07:16'}, u'96c6189f-4cc7-4b74-b05a-ef9d4407bf7f': {u'uuid': u'96c6189f-4cc7-4b74-b05a-ef9d4407bf7f', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/96c6189f-4cc7-4b74-b05a-ef9d4407bf7f', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:07', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:10'}, u'91fe16ea-fb0f-4e56-bdbd-738d99ecb892': {u'uuid': u'91fe16ea-fb0f-4e56-bdbd-738d99ecb892', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/91fe16ea-fb0f-4e56-bdbd-738d99ecb892', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:10:04', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:14'}, u'ca84b980-f81a-4a7e-a403-5826323ca778': {u'uuid': u'ca84b980-f81a-4a7e-a403-5826323ca778', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/ca84b980-f81a-4a7e-a403-5826323ca778', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:10:02', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:13'}, u'ea2a88ea-5c3b-429f-9eca-f5a2b4369964': {u'uuid': u'ea2a88ea-5c3b-429f-9eca-f5a2b4369964', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/ea2a88ea-5c3b-429f-9eca-f5a2b4369964', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:09', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:11'}, u'e8a8dd6c-be5d-421b-938a-019f87997059': {u'uuid': u'e8a8dd6c-be5d-421b-938a-019f87997059', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/e8a8dd6c-be5d-421b-938a-019f87997059', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:52', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:13'}, u'84afdc8d-5ef1-40d6-9699-ceed2038eb51': {u'uuid': u'84afdc8d-5ef1-40d6-9699-ceed2038eb51', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/84afdc8d-5ef1-40d6-9699-ceed2038eb51', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:08:57', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:09'}}, u'message': u'Retrying 1 nodes that failed introspection. Attempt 2 of 3 ', u'introspection_attempt': 2} Exception introspecting nodes: {u'status': u'RUNNING', u'node_uuids': [u'e7b8855a-37c0-429c-9343-13f0e7926572'], u'failed_introspection': [u'e7b8855a-37c0-429c-9343-13f0e7926572'], u'result': None, u'introspected_nodes': {u'948f1601-7f8c-4874-8d44-ce26ae1bc8b8': {u'uuid': u'948f1601-7f8c-4874-8d44-ce26ae1bc8b8', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/948f1601-7f8c-4874-8d44-ce26ae1bc8b8', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:12:35', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:15'}, u'b8e38433-bdb2-4efa-8e9f-1c43f9a15404': {u'uuid': u'b8e38433-bdb2-4efa-8e9f-1c43f9a15404', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/b8e38433-bdb2-4efa-8e9f-1c43f9a15404', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:17', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:12'}, u'e7b8855a-37c0-429c-9343-13f0e7926572': {u'uuid': u'e7b8855a-37c0-429c-9343-13f0e7926572', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/e7b8855a-37c0-429c-9343-13f0e7926572', u'rel': u'self'}], u'finished_at': None, u'state': u'waiting', u'finished': False, u'error': None, u'started_at': u'2018-05-11T20:07:16'}, u'96c6189f-4cc7-4b74-b05a-ef9d4407bf7f': {u'uuid': u'96c6189f-4cc7-4b74-b05a-ef9d4407bf7f', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/96c6189f-4cc7-4b74-b05a-ef9d4407bf7f', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:07', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:10'}, u'91fe16ea-fb0f-4e56-bdbd-738d99ecb892': {u'uuid': u'91fe16ea-fb0f-4e56-bdbd-738d99ecb892', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/91fe16ea-fb0f-4e56-bdbd-738d99ecb892', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:10:04', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:14'}, u'ca84b980-f81a-4a7e-a403-5826323ca778': {u'uuid': u'ca84b980-f81a-4a7e-a403-5826323ca778', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/ca84b980-f81a-4a7e-a403-5826323ca778', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:10:02', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:13'}, u'ea2a88ea-5c3b-429f-9eca-f5a2b4369964': {u'uuid': u'ea2a88ea-5c3b-429f-9eca-f5a2b4369964', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/ea2a88ea-5c3b-429f-9eca-f5a2b4369964', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:09', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:11'}, u'e8a8dd6c-be5d-421b-938a-019f87997059': {u'uuid': u'e8a8dd6c-be5d-421b-938a-019f87997059', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/e8a8dd6c-be5d-421b-938a-019f87997059', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:09:52', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:13'}, u'84afdc8d-5ef1-40d6-9699-ceed2038eb51': {u'uuid': u'84afdc8d-5ef1-40d6-9699-ceed2038eb51', u'links': [{u'href': u'http://192.168.24.1:5050/v1/introspection/84afdc8d-5ef1-40d6-9699-ceed2038eb51', u'rel': u'self'}], u'finished_at': u'2018-05-11T20:08:57', u'state': u'finished', u'finished': True, u'error': None, u'started_at': u'2018-05-11T20:07:09'}}, u'message': u'Retrying 1 nodes that failed introspection. Attempt 2 of 3 ', u'introspection_attempt': 2}
We need to chat with the workflows folks, but this output seems to be by design, at least judging by the source code. I agree that it needs a clean up, but that's an absolutely different issue, even if it seems related. I'd prefer you file it separately (and I personally won't work on it in the near future).
Filed additional bug to address workflow and unnecessary output to user https://bugzilla.redhat.com/show_bug.cgi?id=1578011
It was discussed and agreed that we would verify this based on comment#10
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:2086