Bug 1647149
| Summary: | Attempt to introspect nodes with ovirt-staging driver end insufficient permissions on rhev setup fails with: "Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred". Need better error message. | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Alexander Chuzhoy <sasha> | ||||
| Component: | openstack-ironic | Assignee: | RHOS Maint <rhos-maint> | ||||
| Status: | CLOSED WONTFIX | QA Contact: | |||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 14.0 (Rocky) | CC: | amcleod, bfournie, dcadzow, mburns, sbaker | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2021-05-18 19:53:20 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Created attachment 1502649 [details]
inspector logs
Removing the blocker flag. This was due to lacking permissions on RHEV setup. We need a better message than this error. Unfortunately the docs team doesn't write error messages in the code - happy to assist with wording if need be. The alternative is to document around the error message, which seems suboptimal. Wdyt? Closing for now, RHOS-14 is EOL and the ovirt driver is unsupported. |
Introspection of nodes with staging-ovirt driver fails: Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. Environment: python-ovirt-engine-sdk4-4.2.6-1.el7ev.x86_64 openstack-ironic-staging-drivers-0.10.1-0.20180820161038.39c4e93.el7ost.noarch instack-undercloud-9.4.1-0.20180928005746.15cda5a.el7ost.noarch Steps to reproduce: Deploy undercloud with staging-ovirt driver enabled. openstack baremetal driver list: +---------------------+------------------+ | Supported driver(s) | Active host(s) | +---------------------+------------------+ | idrac | undercloud.local | | ilo | undercloud.local | | ipmi | undercloud.local | | staging-ovirt | undercloud.local | +---------------------+------------------+ Add nodes: (undercloud) [stack@undercloud ~]$ openstack baremetal node list +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ | 96ba2dcc-cf39-44f1-8e61-862035d2e29c | controller-0 | None | power on | manageable | False | | fb11b646-58fd-484f-88eb-843c975438d2 | controller-1 | None | power on | manageable | False | | 5c1a9984-0975-4710-8514-59097c8f82df | controller-2 | None | power on | manageable | False | | c799a7ba-6fad-4238-9225-692476755488 | compute-0 | None | power on | manageable | False | +--------------------------------------+--------------+---------------+-------------+--------------------+-------------+ try to introspect the nodes: (undercloud) [stack@undercloud ~]$ openstack overcloud node introspect --all-manageable --provide Waiting for introspection to finish... Waiting for messages on queue 'tripleo' with no timeout. Introspection of node fb11b646-58fd-484f-88eb-843c975438d2 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node 5c1a9984-0975-4710-8514-59097c8f82df completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node c799a7ba-6fad-4238-9225-692476755488 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node 96ba2dcc-cf39-44f1-8e61-862035d2e29c completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Retrying 4 nodes that failed introspection. Attempt 1 of 3 Introspection of node fb11b646-58fd-484f-88eb-843c975438d2 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node c799a7ba-6fad-4238-9225-692476755488 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node 5c1a9984-0975-4710-8514-59097c8f82df completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node 96ba2dcc-cf39-44f1-8e61-862035d2e29c completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Retrying 4 nodes that failed introspection. Attempt 2 of 3 Introspection of node 5c1a9984-0975-4710-8514-59097c8f82df completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node 96ba2dcc-cf39-44f1-8e61-862035d2e29c completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node fb11b646-58fd-484f-88eb-843c975438d2 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node c799a7ba-6fad-4238-9225-692476755488 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Retrying 4 nodes that failed introspection. Attempt 3 of 3 Introspection of node 96ba2dcc-cf39-44f1-8e61-862035d2e29c completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node fb11b646-58fd-484f-88eb-843c975438d2 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node 5c1a9984-0975-4710-8514-59097c8f82df completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Introspection of node c799a7ba-6fad-4238-9225-692476755488 completed. Status:FAILED. Errors:Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. [u' (HTTP 500) Retry limit reached with 4 nodes still failing introspection {u'status': u'RUNNING', u'node_uuids': [u'96ba2dcc-cf39-44f1-8e61-862035d2e29c', u'fb11b646-58fd-484f-88eb-843c975438d2', u'5c1a9984-0975-4710-8514-59097c8f82df', u'c799a7ba-6fad-4238-9225-6 92476755488'], u'failed_introspection': [u'96ba2dcc-cf39-44f1-8e61-862035d2e29c', u'fb11b646-58fd-484f-88eb-843c975438d2', u'5c1a9984-0975-4710-8514-59097c8f82df', u'c799a7ba-6fad-4238-9225- 692476755488'], u'result': u'Failure caused by error in tasks: send_message\n\n send_message [task_ex_id=0281b14d-32e0-4a50-8396-6afab165eef1] -> Workflow failed due to message status\n [wf_ex_id=436c6e26-974d-4624-8177-fee73b8745ed, idx=0]: Workflow failed due to message status\n', u'introspected_nodes': {u'5c1a9984-0975-4710-8514-59097c8f82df': {u'uuid': u'5c1a9984-0975-4 710-8514-59097c8f82df', u'links': [{u'href': u'http://192.168.0.2:13050/v1/introspection/5c1a9984-0975-4710-8514-59097c8f82df', u'rel': u'self'}], u'finished_at': u'2018-11-06T18:26:07', u's tate': u'error', u'finished': True, u'error': u"Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred.\n[u' (HTTP 500)", u'started_at': u'2018-11-06T18:26: 01'}, u'fb11b646-58fd-484f-88eb-843c975438d2': {u'uuid': u'fb11b646-58fd-484f-88eb-843c975438d2', u'links': [{u'href': u'http://192.168.0.2:13050/v1/introspection/fb11b646-58fd-484f-88eb-843 c975438d2', u'rel': u'self'}], u'finished_at': u'2018-11-06T18:26:01', u'state': u'error', u'finished': True, u'error': u"Failed to set boot device to PXE: Remote error: OVirtError An unknow n exception occurred.\n[u' (HTTP 500)", u'started_at': u'2018-11-06T18:25:58'}, u'96ba2dcc-cf39-44f1-8e61-862035d2e29c': {u'uuid': u'96ba2dcc-cf39-44f1-8e61-862035d2e29c', u'links': [{u'href ': u'http://192.168.0.2:13050/v1/introspection/96ba2dcc-cf39-44f1-8e61-862035d2e29c', u'rel': u'self'}], u'finished_at': u'2018-11-06T18:26:12', u'state': u'error', u'finished': True, u'erro r': u"Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred.\n[u' (HTTP 500)", u'started_at': u'2018-11-06T18:26:01'}, u'c799a7ba-6fad-4238-9225-6924767554 88': {u'uuid': u'c799a7ba-6fad-4238-9225-692476755488', u'links': [{u'href': u'http://192.168.0.2:13050/v1/introspection/c799a7ba-6fad-4238-9225-692476755488', u'rel': u'self'}], u'finished_ at': u'2018-11-06T18:26:16', u'state': u'error', u'finished': True, u'error': u"Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred.\n[u' (HTTP 500)", u' started_at': u'2018-11-06T18:26:01'}}, u'message': u'Retrying 4 nodes that failed introspection. Attempt 1 of 3 ', u'introspection_attempt': 1} Exception introspecting nodes: {u'status': u'RUNNING', u'node_uuids': [u'96ba2dcc-cf39-44f1-8e61-862035d2e29c', u'fb11b646-58fd-484f-88eb-843c975438d2', u'5c1a9984-0975-4710-8514-59097c8f82d f', u'c799a7ba-6fad-4238-9225-692476755488'], u'failed_introspection': [u'96ba2dcc-cf39-44f1-8e61-862035d2e29c', u'fb11b646-58fd-484f-88eb-843c975438d2', u'5c1a9984-0975-4710-8514-59097c8f82 df', u'c799a7ba-6fad-4238-9225-692476755488'], u'result': u'Failure caused by error in tasks: send_message\n\n send_message [task_ex_id=0281b14d-32e0-4a50-8396-6afab165eef1] -> Workflow fai led due to message status\n [wf_ex_id=436c6e26-974d-4624-8177-fee73b8745ed, idx=0]: Workflow failed due to message status\n', u'introspected_nodes': {u'5c1a9984-0975-4710-8514-59097c8f82d f': {u'uuid': u'5c1a9984-0975-4710-8514-59097c8f82df', u'links': [{u'href': u'http://192.168.0.2:13050/v1/introspection/5c1a9984-0975-4710-8514-59097c8f82df', u'rel': u'self'}], u'finished_a t': u'2018-11-06T18:26:07', u'state': u'error', u'finished': True, u'error': u"Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred.\n[u' (HTTP 500)", u's tarted_at': u'2018-11-06T18:26:01'}, u'fb11b646-58fd-484f-88eb-843c975438d2': {u'uuid': u'fb11b646-58fd-484f-88eb-843c975438d2', u'links': [{u'href': u'http://192.168.0.2:13050/v1/introspect ion/fb11b646-58fd-484f-88eb-843c975438d2', u'rel': u'self'}], u'finished_at': u'2018-11-06T18:26:01', u'state': u'error', u'finished': True, u'error': u"Failed to set boot device to PXE: Rem ote error: OVirtError An unknown exception occurred.\n[u' (HTTP 500)", u'started_at': u'2018-11-06T18:25:58'}, u'96ba2dcc-cf39-44f1-8e61-862035d2e29c': {u'uuid': u'96ba2dcc-cf39-44f1-8e61-86 2035d2e29c', u'links': [{u'href': u'http://192.168.0.2:13050/v1/introspection/96ba2dcc-cf39-44f1-8e61-862035d2e29c', u'rel': u'self'}], u'finished_at': u'2018-11-06T18:26:12', u'state': u'er ror', u'finished': True, u'error': u"Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred.\n[u' (HTTP 500)", u'started_at': u'2018-11-06T18:26:01'}, u'c79 9a7ba-6fad-4238-9225-692476755488': {u'uuid': u'c799a7ba-6fad-4238-9225-692476755488', u'links': [{u'href': u'http://192.168.0.2:13050/v1/introspection/c799a7ba-6fad-4238-9225-692476755488', u'rel': u'self'}], u'finished_at': u'2018-11-06T18:26:16', u'state': u'error', u'finished': True, u'error': u"Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred.\n[u' (HTTP 500)", u'started_at': u'2018-11-06T18:26:01'}}, u'message': u'Retrying 4 nodes that failed introspection. Attempt 1 of 3 ', u'introspection_attempt': 1} ironic-inspector.log has repeating entries of: 2018-11-06 13:28:07.291 1 ERROR ironic_inspector.node_cache [-] [node: 5c1a9984-0975-4710-8514-59097c8f82df state starting] Processing the error event because of an exception <class 'ironic_inspector.utils.Error'>: Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred. 2018-11-06 13:28:12.275 1 ERROR ironic_inspector.utils [-] [node: c799a7ba-6fad-4238-9225-692476755488 state starting] Failed to set boot device to PXE: Remote error: OVirtError An unknown exception occurred.