Description of problem: After updating undercloud from 16.1 -> 16.2 or from 13 -> 16.2 directly, "Power State" of baremetal nodes are set to "None" and nodes are automatically moved to maintenance mode. I am using virtual BMC in this test. I am not sure if this problem is specific to use vbmc or not. After updating to 16.2 $ openstack baremetal node list +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ | 1c902016-8a8a-498a-835c-bb787a8ccb80 | controller-1 | 31e1f76d-2848-495b-91bd-82cb901d4336 | None | active | True | | b3bf8d60-9ee2-4040-9db2-b7446d700005 | controller-2 | 39689891-d41d-4dbc-b327-2743b70fb080 | None | active | True | | 8442907a-40d0-4b2f-a24b-4f01a48cf916 | controller-3 | 910cc393-0bb6-4c3a-98bd-6bffc54aa253 | None | active | True | | bbe87cdb-abf8-437b-ba11-ffc0b77a3703 | compute-1 | 0def0062-e778-4b96-8ccc-2489cfd854a5 | None | active | True | | d27aa7d3-ea50-432a-b416-8b315dc8f302 | compute-2 | 4e603576-5e30-4281-999e-488e3a3c22d2 | None | active | True | | eeae8efd-95ab-49d0-9b6e-c41098f00541 | ceph-1 | 44566b9f-c070-4c4e-ae05-c93d4563a76e | None | active | True | | f84e8bd7-f181-4644-86e4-de476cb90430 | ceph-2 | 18a102bc-7489-4d48-9f1c-22d97abc67a1 | None | active | True | | a8fa3c05-6557-4609-8649-003cc2ee26ec | ceph-3 | e9d802fc-b0fc-4dbe-8408-263a7df1de6d | None | active | True | $ openstack baremetal node show compute-1 +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | allocation_uuid | None | | automated_clean | None | | bios_interface | no-bios | | boot_interface | ipxe | | chassis_uuid | None | | clean_step | {} | | conductor | undercloud.redhat.local | | conductor_group | | | console_enabled | False | | console_interface | ipmitool-socat | | created_at | 2021-09-20T16:18:31+00:00 | | deploy_interface | iscsi | | deploy_step | {} | | description | None | | driver | ipmi | | driver_info | {'deploy_kernel': 'file:///var/lib/ironic/httpboot/agent.kernel', 'rescue_kernel': 'file:///var/lib/ironic/httpboot/agent.kernel', 'deploy_ramdisk': 'file:///var/lib/ironic/httpboot/agent.ramdisk', 'rescue_ramdisk': 'file:///var/lib/ironic/httpboot/agent.ramdisk', 'ipmi_address': '192.168.1.8', 'ipmi_password': '******', 'ipmi_username': 'admin'} | | driver_internal_info | {'is_whole_disk_image': False, 'deploy_boot_mode': 'bios', 'deploy_steps': None, 'last_power_state_change': '2021-09-20T16:42:31.171346', 'agent_version': '5.0.4.dev8', 'agent_last_heartbeat': '2021-09-20T16:38:01.613741', 'root_uuid_or_disk_id': '670cf845-41e8-475a-af33-ec2750510daf'} | | extra | {'hardware_swift_object': 'extra_hardware-bbe87cdb-abf8-437b-ba11-ffc0b77a3703'} | | fault | power failure | | inspect_interface | inspector | | inspection_finished_at | None | | inspection_started_at | None | | instance_info | {'image_source': '5238ab40-0293-4847-a0b4-92f4c2254304', 'root_gb': '40', 'swap_mb': '0', 'display_name': 'compute-1', 'vcpus': '1', 'nova_host_id': 'undercloud.redhat.local', 'memory_mb': '4096', 'local_gb': '99', 'configdrive': '******'} | | instance_uuid | 0def0062-e778-4b96-8ccc-2489cfd854a5 | | last_error | During sync_power_state, max retries exceeded for node bbe87cdb-abf8-437b-ba11-ffc0b77a3703, node state None does not match expected state 'power on'. Updating DB state to 'None' Switching node to maintenance mode. Error: An exclusive lock is required, but the current context has a shared lock. | | maintenance | True | | maintenance_reason | During sync_power_state, max retries exceeded for node bbe87cdb-abf8-437b-ba11-ffc0b77a3703, node state None does not match expected state 'power on'. Updating DB state to 'None' Switching node to maintenance mode. Error: An exclusive lock is required, but the current context has a shared lock. | | management_interface | ipmitool | | name | compute-1 | | network_interface | flat | | owner | None | | power_interface | ipmitool | | power_state | None | | properties | {'capabilities': 'boot_option:local', 'local_gb': '99', 'cpus': '4', 'cpu_arch': 'x86_64', 'memory_mb': '8144'} | | protected | False | | protected_reason | None | | provision_state | active | | provision_updated_at | 2021-09-20T16:42:51+00:00 | | raid_config | {} | | raid_interface | no-raid | | rescue_interface | agent | | reservation | None | | resource_class | baremetal | | storage_interface | noop | | target_power_state | None | | target_provision_state | None | | target_raid_config | {} | | traits | [] | | updated_at | 2021-09-22T01:58:47+00:00 | | uuid | bbe87cdb-abf8-437b-ba11-ffc0b77a3703 | | vendor_interface | ipmitool | +------------------------+--------------------------------------------------------------------------------------------------------------------------------------- Can be fixed by setting power on and maintenance to false manually. $ for i in compute-2 controller-1 controller-2 controller-3 ceph-1 ceph-2 ceph-3; do openstack baremetal node power on $i; openstack baremetal node maintenance unset $i; done (undercloud) [stack@undercloud ~]$ openstack baremetal node list +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ | 1c902016-8a8a-498a-835c-bb787a8ccb80 | controller-1 | 31e1f76d-2848-495b-91bd-82cb901d4336 | power on | active | False | | b3bf8d60-9ee2-4040-9db2-b7446d700005 | controller-2 | 39689891-d41d-4dbc-b327-2743b70fb080 | power on | active | False | | 8442907a-40d0-4b2f-a24b-4f01a48cf916 | controller-3 | 910cc393-0bb6-4c3a-98bd-6bffc54aa253 | power on | active | False | | bbe87cdb-abf8-437b-ba11-ffc0b77a3703 | compute-1 | 0def0062-e778-4b96-8ccc-2489cfd854a5 | power on | active | False | | d27aa7d3-ea50-432a-b416-8b315dc8f302 | compute-2 | 4e603576-5e30-4281-999e-488e3a3c22d2 | power on | active | False | | eeae8efd-95ab-49d0-9b6e-c41098f00541 | ceph-1 | 44566b9f-c070-4c4e-ae05-c93d4563a76e | power on | active | False | | f84e8bd7-f181-4644-86e4-de476cb90430 | ceph-2 | 18a102bc-7489-4d48-9f1c-22d97abc67a1 | power on | active | False | | a8fa3c05-6557-4609-8649-003cc2ee26ec | ceph-3 | e9d802fc-b0fc-4dbe-8408-263a7df1de6d | power on | active | False | | 8c35a2d7-a6e9-4003-a530-4453a0edf007 | compute-3 | 4030321f-5430-4e3f-a504-839f1ecbc8e6 | power on | active | False | +--------------------------------------+--------------+--------------------------------------+-------------+--------------------+-------------+ Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
We have a fix incoming for this issue *** This bug has been marked as a duplicate of bug 2007268 ***