Description of problem:
Issue: When deploying overcloud for the first time pxe is used and deployment works. But if you delete overcloud and try deployment again, pxe is not put first and because there is still an OS on the nodes, they boot on it and overcloud deployment never work.

Nodes: Dell PowerEdge R740
iDRAC Firmware version used: 3.21.26.22 (latest available, came out a few days ago)

Nodes were using UEFI but now are Legacy BIOS.
iDrac only is being used. Can't use ipmitool because of security constraints.

Have tried deployment with the following:
openstack baremetal node boot device set --persistent compute-0 pxe


boot_mode:uefi was set on the nodes when using UEFI boot mode.

Version-Release number of selected component (if applicable):
openstack-ironic-api-10.1.6-4.el7ost.noarch                 Mon Feb 18 22:39:43 2019
openstack-ironic-common-10.1.6-4.el7ost.noarch              Mon Feb 18 22:39:39 2019
openstack-ironic-conductor-10.1.6-4.el7ost.noarch           Mon Feb 18 22:39:48 2019
openstack-ironic-inspector-7.2.1-5.el7ost.noarch            Mon Feb 18 22:40:02 2019
openstack-ironic-staging-drivers-0.9.1-1.el7ost.noarch      Mon Feb 18 22:39:56 2019


How reproducible:


Steps to Reproduce:
1. Delete overcloud
2. Try deployment again
3. Deployment doesn't work because nodes never boot on PXE.

Actual results:
Subsequent deployment of the overcloud fails because ironic is not able to boot on PXE (boots on disks instead).

Expected results:
Ironic should be able to set boot order to pxe when doing subsequent deployment of the overcloud.

Additional info: