Description of problem:

Ironic is setting maintenance mode enabled on all registered nodes after upgrading from 16.1 to 16.2.

Environment summary:

Environment was a 16.1 DCN deployment with Ironic in the Overcloud which I performed an upgrade to 16.2 on.  While upgrading the overcloud, it was noticed all of the baremetal nodes registered to director were in maintenance mode. At the conclusion of the upgrade, I attempted to deploy a baremetal instance in the overcloud and saw I was getting NoValidHost Found.

Ironic in the overcloud had it's baremetal nodes set to maintenance true.  I unset them and they were immediately changed back to maintenance mode true.

I checked Ironic in the undercloud and noted the same behaviour.

ironic-conductor.log shows:

2021-10-06 10:39:15.653 7 ERROR ironic.conductor.manager [req-9d2a85ef-2fca-497b-8ef2-7aa28f10386d - - - - -] During sync_power_state, max retries exceeded for node cad0d948-b462-4d11-af75-65f15ee6ef23, node state None does not m│·······
atch expected state 'None'. Updating DB state to 'None' Switching node to maintenance mode. Error: An exclusive lock is required, but the current context has a shared lock.: ironic.common.exception.ExclusiveLockRequired: An exclu│·······
sive lock is required, but the current context has a shared lock. 



I confirmed I can use ipmitool from within the ironic conductor container on Director using the credentials registered with ironic and successfully obtain a power status on the above referenced node.

Version-Release number of selected component (if applicable):
16.2

How reproducible:


Steps to Reproduce:
1. Upgrade environment from 16.1 to 16.2
2.
3.

Actual results:


Expected results:


Additional info: