it's possible this sync is coming from nova.

in /etc/ironic/ironic.conf on the undercloud we set:
force_power_state_during_sync=False
so ironic shouldn't be enforcing any power state

can you attach output of the following:

ironic node-list
nova list

and also the following logs:
/var/log/nova/*
/var/log/ironic/*

(In reply to James Slagle from comment #3)
> it's possible this sync is coming from nova.
> 
> in /etc/ironic/ironic.conf on the undercloud we set:
> force_power_state_during_sync=False
> so ironic shouldn't be enforcing any power state
> 
> can you attach output of the following:
> 
> ironic node-list
> nova list
> 
> and also the following logs:
> /var/log/nova/*
> /var/log/ironic/*

Probably I can see in the nova-compute.log, following:
2015-07-21 17:11:06.160 31535 INFO nova.compute.manager [-] [instance: cd2cec9f-f913-4e
fe-a65f-c64376c8f741] During _sync_instance_power_state the DB power_state (4) does not
 match the vm_power_state from the hypervisor (1). Updating power_state in the DB to ma
tch the hypervisor.
2015-07-21 17:11:06.178 31535 DEBUG oslo_concurrency.lockutils [-] Lock "d83a4227-4a29-
40ec-a08c-65f839bc6e8d" released by "query_driver_power_state_and_sync" :: held 0.259s 
inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:456
2015-07-21 17:11:06.178 31535 DEBUG oslo_concurrency.lockutils [-] Lock "bb016181-0435-
46f6-add6-ee1c96faa8b6" released by "query_driver_power_state_and_sync" :: held 0.260s 
inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:456
2015-07-21 17:11:06.313 31535 WARNING nova.compute.manager [-] [instance: cd2cec9f-f913
-4efe-a65f-c64376c8f741] Instance is not stopped. Calling the stop API. Current vm_stat
e: stopped, current task_state: None, original DB power_state: 4, current VM power_stat
e: 1
2015-07-21 17:11:06.314 31535 DEBUG nova.compute.api [-] [instance: cd2cec9f-f913-4efe-
a65f-c64376c8f741] Going to try to stop instance force_stop /usr/lib/python2.7/site-pac
kages/nova/compute/api.py:1864

and the ironic can just see that RPC call was made:
2015-07-21 17:11:06.733 21582 DEBUG ironic.conductor.manager [-] RPC change_node_power_state called for node 7d918d90-7583-4482-b781-0bf3b0f05de8. The desired new state is power off. change_node_power_state /usr/lib/python2.7/site-packages/ironic/conductor/manager.py:431

The ironic setting - force_power_state_during_sync=False seems to be useless provided nova makes the power state sync

Output of reguested commands (at the moment of compute-1 being powered on but still shown as SHUTOFF in nova list)

nova list
+--------------------------------------+------------------------+---------+------------+-------------+---------------------+
| ID                                   | Name                   | Status  | Task State | Power State | Networks            |
+--------------------------------------+------------------------+---------+------------+-------------+---------------------+
| d83a4227-4a29-40ec-a08c-65f839bc6e8d | overcloud-compute-0    | ACTIVE  | -          | Running     | ctlplane=192.0.2.10 |
| cd2cec9f-f913-4efe-a65f-c64376c8f741 | overcloud-compute-1    | SHUTOFF | -          | Shutdown    | ctlplane=192.0.2.12 |
| bb016181-0435-46f6-add6-ee1c96faa8b6 | overcloud-controller-0 | ACTIVE  | -          | Running     | ctlplane=192.0.2.9  |
+--------------------------------------+------------------------+---------+---------

ironic node-list+--------------------------------------+------+--------------------------------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID                        | Power State | Provision State | Maintenance |
+--------------------------------------+------+--------------------------------------+-------------+-----------------+-------------+
| 7d918d90-7583-4482-b781-0bf3b0f05de8 | None | cd2cec9f-f913-4efe-a65f-c64376c8f741 | power on    | active          | False       |
| b3076d11-c2cd-4214-88ff-95de25158cc5 | None | bb016181-0435-46f6-add6-ee1c96faa8b6 | power on    | active          | False       |
| 70e32afd-63fe-4306-b0eb-3b90dce0b078 | None | None                                 | power off   | available       | False       |
| e96c762a-b2df-44df-84a9-2d7579fc35e5 | None | d83a4227-4a29-40ec-a08c-65f839bc6e8d | power on    | active          | False       |
+--------------------------------------+------+--------------------------------------+

so everything looks like it's doing what it's supposed to. The ironic setting - force_power_state_during_sync=False is not useless, what makes you think so? It seems to be doing exactly what is asked.

I'm not sure what you're requesting. Would you like to see nova configured not to sync the instance state? That would mean setting sync_power_state_interval to -1 in /etc/nova/nova.conf, but I'm not convinced that's the right thing to do here.

(In reply to James Slagle from comment #5)
> so everything looks like it's doing what it's supposed to. The ironic
> setting - force_power_state_during_sync=False is not useless, what makes you
> think so? It seems to be doing exactly what is asked.
Yes It does what It is asked for but the result of the environment is the opposite - even I set the ironic the way I want I need to keep in mind the related settings in nova. Two places are set to behave in opposite way.
> 
> I'm not sure what you're requesting. Would you like to see nova configured
> not to sync the instance state? That would mean setting
> sync_power_state_interval to -1 in /etc/nova/nova.conf, but I'm not
> convinced that's the right thing to do here.
Well in my opinion yes, the same reason not to set ironic to sync power state is valid for doing the same for nova. Moreover as I am testing it, it seems that nova "will win" always - even though I set force_power_state_during_sync=true, and set the node to ON state through ironic, nova will force the state It has in DB i.e SHUTOFF.
Anyway I am not an expert and observed this by accident when doing something different (which is a proof the bug came from real experiences though), so whatever needs to be decided, I am expressing my opinion.

This is terrible wrt HA, once nova assumes undercloud instance is SHUTOFF nothing will change its mind unless I got and directly start the instance using nova cli command, I guess It causes all my HA tests to fail.

this is something to do with nova or ironic syncing power state. assigning to ironic team

(Copying from the email I sent today)

Both Ironic and Nova has mechanisms to sync the state of the instance to according to what is registered in their database. (I will try to describe both, it would be good to have someone from Nova to review my understanding of it).

** Ironic **

Ironic has a configuration option called "force_power_state_during_sync" [0] that allows Ironic to act in active or passive more, for example, if that option is set to True Ironic will force the power state to the node, that means that, if the node is "powered off" but in the Ironic database it's marked as "powered on" Ironic will power on that node. When that option is set to False, Ironic will just update the database according to the real state of the node. So, if the node is "powered off" but in the Ironic database it's marked as "powered on" Ironic will just update the database to "powered off" and will not do anything with the node.

By default in director we this option in Ironic is always set to False [1].


** Nova **

Nova has a configuration option called "sync_power_state_interval" this is the interval to run a periodic task [2] which will run this code here [3]. Basically it will compare the instance state (what's registered in the database) with the VM state (which is the Ironic node state. In the Nova POV an instance in Ironic is just a "vm") and if the states are different the nova periodic task will try to force what's registered instance state onto the vm state.

Different from Ironic I don't think that nova has a way to make it act in passive mode and just update the instance state according to the VM state. But it's possible to disable that periodic task by setting it's value to a negative value, e.g:


[DEFAULT]
 ...
 sync_power_state_interval=-1


The problem with that approach is that the instance in Nova will be marked as SHUTDOWN even tho it's actually running, this can be corrected manually by doing a "nova start <instance name>".


So, possible solutions would be as you suggested to have the fencing happening at the nova layer by using the nova start/stop commands. I don't fully grasp the problem Fabio raises with this approach, because powering the node on/off via IPMI command or via nova start/stop is basically the same (since underneath Ironic will issue IPMI commands).

Another way I can think of, is extending that "sync_power_state_interval" configuration option in Nova to run in passive mode and just change the database instead of acting on the VM state directly.

[0] https://github.com/openstack/ironic/blob/9b48134615aa6a7406aa1395f175ee1ddec42090/ironic/conductor/manager.py#L112-L117
[1] https://github.com/rdo-management/instack-undercloud/blob/31f208b5556efb4948576d8335bfb4b8643db593/elements/puppet-stack-config/puppet-stack-config.yaml.template#L189
[2] https://github.com/openstack/nova/blob/c23870b362a63b046c1ae628313931369b54b6e9/nova/compute/manager.py#L5800-L5802
[3] https://github.com/openstack/nova/blob/c23870b362a63b046c1ae628313931369b54b6e9/nova/compute/manager.py#L5920-L5964

Hope that helps,
Lucas

*** Bug 1246053 has been marked as a duplicate of this bug. ***

Hi @Fabio,

Can you please take a look at the possible options we have to fix this and comment on them?

Thank you,
Lucas

(In reply to Lucas Alvares Gomes from comment #13)
> Hi @Fabio,
> 
> Can you please take a look at the possible options we have to fix this and
> comment on them?
> 
> Thank you,
> Lucas

Please do not reassign bugs to people just to ask information. set needinfo for that purpose.
(In reply to Lucas Alvares Gomes from comment #11)

> So, possible solutions would be as you suggested to have the fencing
> happening at the nova layer by using the nova start/stop commands. I don't
> fully grasp the problem Fabio raises with this approach, because powering
> the node on/off via IPMI command or via nova start/stop is basically the
> same (since underneath Ironic will issue IPMI commands).

What problem is it exactly? I am missing a reference here. Is it something that's been discussed in an email thread?

Lucas, will you please write the doc text for this?

@Fabio: Context is in the comments. In summary "Nova and Pacemaker are conflicting when sync'ing the power state of the instance."

(In reply to Jaromir Coufal from comment #16)
> @Fabio: Context is in the comments. In summary "Nova and Pacemaker are
> conflicting when sync'ing the power state of the instance."

Yes there has been email discussion that was not captured here in the BZ.

replying to comment #11 (copy paste from email thread):

=========

Lucas wrote:
> Fabio wrote:
>
> fencing has to be reliable and precise.
>
> nova has a bunch of problems in that area. First of all a shutdown of an
> instance (that being overcloud/undercloud/normal instance) can fail for
> several more reasons than direct IPMI, second we depend on nova
> scheduler to act upon the request. If the undercloud is busy, the
> request will take longer to be executed, and as a consequence it will
> delay the time it takes to complete fencing operation and start
> recovering operations. This is a major hit towards the uptime/SLA of a
> high available solution.
>
> That is to do a off/status/on check on IPMI, it will now requiring
> passing by several layers (taking a much longer time) and every single
> one of them represent a potential single point of failure in the chain
> to start recovering a cluster.
>
> high availability is about removing single point of failures and right
> now, the undercloud is a major one (not being HA itself atm).
>
> I am all into helping syncing the DB state for undercloud ironic and
> nova, but fence_nova is not yet a viable option.
>

Ok fair enough, I tend to agree with this. But that doesn't fit the current design
if we leave that option in Nova enabled to sync the VM state according to the instance
state, because like this we did end up with two services power controlling the same
nodes.

So this narrows down the options we have:

1. Extend nova to be able to just sync the state of the VM and update the instance state

2. Disable that sync in nova and document issuing a "nova start" command in order to mark
the instance as running instead of shutdown.

(Anything else?)

===============

So the problem really is:

* If ironic driver tells Nova that the instance is down, and Nova believes the instance is up, nova will stop the instance. The inverse is not true though, (and this is the crux of the issue). If the driver tells Nova that the instance is RUNNING, Nova will try to shut it down until the user volountarily starts the instance.

This is the safest thing to do. We don't want to be starting instances that should not be up. Actually I would expect more of warning in the logs than just "Instance not stopped" (but it should be).

A tempting idea might be to record what was the reason for our current powerstate (user or driver sync) and then allow driver sync to override itself, which would solve the particular problem discussed here.

Ultimately though - this is at odds with Nova's design, and is a good example of where the "baremetal nodes as VMs" abstractions starts to leak. Nova needs to be fully in control of machines it is managing, module some acceptable edge cases (user calls shutdown inside the VM). This is a reasonable thing to assume for VMs but for managing physical infrastructure especially with HA in mind, this seems to break down. The solution is not clear cut, but taking into account the current direction of the upstream Nova project, it is unlikely that it will be easily found within Nova.

We could attempt to propose the idea presented above to the upstream community, but it is not in any way trivial to implement and is unlikely to be met with a lot of sympathy upstream (but let's see).

From the POV of this bug - the timeline we are talking about is the N release at best (meaning OSP 10), so I propose to clone this bug (I'll leave that up to the PM) and target it appropriately with upstream timelines in mind.

For RHOS 8 this is squarely in the WONTFIX category IMHO

I am reopening the bug just to keep track of it. We shouldn´t close bugs that we plan or consider to fix in future, but rather simply re-target them to the appropriate release, otherwise they will get lost.

that said, if the fix is not in nova, or it will be in some other components, we can always reassign it.

*** Bug 1300450 has been marked as a duplicate of this bug. ***

I experienced this issue as well:

I did an overcloud deployment, enabled fencing, and then needed to update neutron/plugin.ini to put in the vlan ranges that were not populated due to another bug.  I updated the file on all 3 controller nodes and rebooted the nodes one at a time.  The nodes kept being powered off, and I ended up with an HA openstack cluster with 0 controllers.

Investigation showed that the nodes were marked as shutdown in nova.  nova-compute.log shows nova shutting the nodes down.  Further investigation found the sync_power_state_interval parameter, which I am in the process of trying to set to -1 so we can proceed with testing.

This really is unacceptable to have director let the controller nodes get into a state where they can't be brought up.  We really need a fix for this ASAP, though I'm hoping that setting sync_power_state_interval to -1 will provide a temporary workaround at least.

Note that the scenario above has nothing to do with fencing.

Can you tell us how you were rebooting the nodes?

Sure - ssh into a controller node and: shutdown -h now

Wait for a few minutes.  Do a "nova list".  The node will be marked SHUTOFF.  If you power on the node via the iDRAC console at this point, it will be powered off again within 10 minutes.

I tried "nova start <uuid>" on each of the controllers.  This rippled through to ironic & the nodes were brought up and stayed up.

Chris,  did the other workaround from comment 21 work?

sync_power_state_interval=-1

Just tested this, and the controller nodes have stayed up for over 10 minutes.  As noted above, the controllers are listed as SHUTDOWN in the "nova list" output.  IMHO this inaccuracy is much preferred over the controllers being powered off every 10 minutes!

We suffer the same exact problem during our HA test. We don't have fencing but we simulate a crash on one controller, and when we boot it again (without using Director), director start to kill the machine every 10min. We are looking for a workaround and and a eventual solution.

We will like to apply the  sync_power_state_interval=-1  workaround but we are afraid that could impact with our future deployments. Anyone that could confirm if the workaround has other drawbacks?

Hi,
As Jorge, who's hit this issue in test env, we managed to workaround as having ironic and nova with a different state. The environment is currently stabilized that includes having the  setting the "sync_power_state_interval=-1" in place.

Now, What would the proper recover procedure be? The Chris c#24 way? 
Or the options that fabio mentioned as option 1 at #17? 

Having a common ground approach it would hep us both guide the customer in the right direction and create a KB article for handling this in the future.

Thanks,
Ondrej

(In reply to Ondrej from comment #28)
> Hi,
> As Jorge, who's hit this issue in test env, we managed to workaround as
> having ironic and nova with a different state. The environment is currently
> stabilized that includes having the  setting the
> "sync_power_state_interval=-1" in place.
> 
> Now, What would the proper recover procedure be? The Chris c#24 way? 
> Or the options that fabio mentioned as option 1 at #17? 
> 

So option 1 is not viable for the release being tested here as explained in comment #18.

The problem with disabling the periodic sync is that Nova will not notice the instance is SHUTOFF and it will keep it as ACTIVE even when the node loses power and Ironic does notice it.

This may be fine as long as it's well documented and the user knows to check Ironic for the real state of things (but is obviously far from ideal). So we would say that Nova list can get out of sync and checking the Ironics' view on things is necessary to get the full picture and potentially update Nova manually. This may however interfere with some assumptions made by the HA infra so it's probably worth testing out.

In case of fencing though, assuming that pacemaker is in control of power-cycling the node, I wonder if it would be possible to automate doing stop/start in Nova in addition to fencing the node so as to keep things in sync and make the situation a bit better.

HA team - how would the above affect your bits?

(In reply to Nikola Dipanov from comment #29)
> (In reply to Ondrej from comment #28)
> > Hi,
> > As Jorge, who's hit this issue in test env, we managed to workaround as
> > having ironic and nova with a different state. The environment is currently
> > stabilized that includes having the  setting the
> > "sync_power_state_interval=-1" in place.
> > 
> > Now, What would the proper recover procedure be? The Chris c#24 way? 
> > Or the options that fabio mentioned as option 1 at #17? 
> > 
> 
> So option 1 is not viable for the release being tested here as explained in
> comment #18.
> 
> The problem with disabling the periodic sync is that Nova will not notice
> the instance is SHUTOFF and it will keep it as ACTIVE even when the node
> loses power and Ironic does notice it.
> 
> This may be fine as long as it's well documented and the user knows to check
> Ironic for the real state of things (but is obviously far from ideal). So we
> would say that Nova list can get out of sync and checking the Ironics' view
> on things is necessary to get the full picture and potentially update Nova
> manually. This may however interfere with some assumptions made by the HA
> infra so it's probably worth testing out.

Currently HA (fencing or pacemaker) doesn´t talk to nova or ironic in the undercloud, so whatever setting you do there, doesn´t interfere with HA operations. What matters is that there is only one power manager and with fencing enable it has to be pacemaker.

Please note that if you do a graceful shutdown of a node, fencing won´t (shouldn´t) kick in and pacemaker will just recognize that as a node is being shutdown. So any normal admin operations will be fine as long as at least 2 out of 3 nodes are alive.

> 
> In case of fencing though, assuming that pacemaker is in control of
> power-cycling the node, I wonder if it would be possible to automate doing
> stop/start in Nova in addition to fencing the node so as to keep things in
> sync and make the situation a bit better.
> 
> HA team - how would the above affect your bits?

In pacemaker it is possible to chain fencing mechanisms.

I can see the option to do:

1) fence_undercloud_notify action off -> we are about to poweroff this node hard, please stop monitoring

2) fence_ipmi (or ucs or power fencing goes here) off -> on

3) fence_undercloud_notify action on -> we have powercycled the node, life goes on

That said, the undercloud_notify could potentially also talk to ironic (if there is an exposed API) and tell ironic to keep its hands off the power plugs for a bit.

My only concern is that we will introduce another layer of complexity in fencing, that´s not very different from using a fence_nova_undercloud and asking nova to shutdown the node. The notification needs to be 100% reliable otherwise we are back at square 0.

After some further discussion with both Ironic and HA teams, I think we have a more clear understanding of the impact on different components. I'll try to sum up the angles considered and present a workaround we will likely want to implement and document the caveats that go with it.

Setting sync_power_state_interval to -1 means that in practice Nova will not have correct status, as it will not be syncing any changes such as loss of power to an bare metal node. As has been discussed in comment #11, Ironic will have the correct view of whether the Ironic managed bare-metal node is running or not. This inaccuracy with statuses will not impact HA infra in any way.

Based on the above - the best workaround seems to be to

1) Set sync_power_state_interval to -1 by default.
2) Document (as a release note and as a KB article) that nova list is likely to show incorrect results for the undercloud, and that it is necessary to check the states in Ironic to get the accurate view.

I will move the bug to instack-undercloud so that the defaults are changed as described above. In the meantime, as a workaround - feel free to set the config option as described above.

Nikola, Fabio,
thanks for your comments, now there is a situation on  Jorge Martinez from comment #27) need to be addressed.

From what I undertand, the key point in their original issue the nova showed transitioned from  active to shutoff ( more precise : status SHUTOFF | - power state | Shutdown) and from ironic showed  power status = power off, status =active and maintenance =True 

their main concern is that the person doing the task a that moment, never realized about the shutdown status and from their expectation was nova not changing the status as still marked active (at the moment of starting the controller instance).

Then the Nikola explains in his comment 18 confirm the rest of the actions. May be Jorge can add any additional or observation about what I interpreted in their case if I missing some part of their question.

Thanks Pablo, yes I am agree with Nikola on comment 18,

I think Nova should not try to enforce a status that was never requested in the first place by the user on any means (operator or fencing).

The SHUTOFF status was set by Nova itself in order to reflect the real status that ironic reports, which totally make sense for me. But then, it "forgets" that this status was not requested in the first place,... that's Nova mistake in my humble opinion.

As Nikola says, maybe this type of particular transition can be flagged by Nova so it's able to react different on the two possible scenarios:

- status was requested from user somehow in the past => enforce it against ironic
- status was set by itself => transit again to the status that match ironic power state instead

I think that what needs to happen in the long run is to have nova add a config option like ironic that disables forcing the power state to match.  Disabling the sync will result in this happening as a consequence but also leaves the DB out of sync.  

Ironic has an option force_power_state_during_sync.  When false (how it's set by director), this allows it to update it's DB to match the actual status by updating it's DB and not changing the actual power state.  IOW, if ironic thinks node0 is down, checks that it's powered on and sees it up, it simply updates it's database to say up.  When true, it will make the machine match the database.

(In reply to Jorge Martinez from comment #33)
> 
> As Nikola says, maybe this type of particular transition can be flagged by
> Nova so it's able to react different on the two possible scenarios:
> 
> - status was requested from user somehow in the past => enforce it against
> ironic
> - status was set by itself => transit again to the status that match ironic
> power state instead

This might be a solution we could propose upstream (even though one could reasonably doubt how useful this will be in the general case), but is almost definitely not viable for RHOSP 7

I agree with Mike's assessment in comment 34.  Nova should be modified to work like ironic, which has 2 options:
1. Force the nodes into the power state that's in the Nova DB (Current behavior)
2. Update the DB with the actual power state of the node (desired behavior in the scenario above)

Nikola,

For undercloud operations where we are doing just baremetal provisioning, are we safe doing just set sync_power_state_interval to -1 ?

are there any consequences beyond the nova db being out of sync?

Based on comment #26 and my review - it seems safe.

It would be good to test that restarting the nova-compute service does not attempt to rectify this in any way (although from code inspection it appears that it should be fine).

I just want to add that this problem affects also the instance HA support.
If you follow the steps of the knowledge base here [1] and you try to test by hanging a compute node, then you will find yourself in a situation in which the compute node gets correctly fenced, but when the machine turn up again it gets power-offed by Nova.
So, changing the value sync_power_state_interval to -1 in nova.conf is good to solve also this problem.
Funny thing is that even if the power off is driven by nova, nova itself and ironic sees the node as up and running.

[1] https://access.redhat.com/articles/1544823

Hi,

> Funny thing is that even if the power off is driven by nova, nova itself and
> ironic sees the node as up and running.

That doesn't sound like the correct behavior, the power state can get off for some time since the sync runs in a periodic task but, Ironic will correct the state eventually*.

* See the "sync_power_state_interval" configuration option, it defaults to 60 seconds.

(In reply to Lucas Alvares Gomes from comment #40)
> Hi,
> 
> > Funny thing is that even if the power off is driven by nova, nova itself and
> > ironic sees the node as up and running.
> 
> That doesn't sound like the correct behavior, the power state can get off
> for some time since the sync runs in a periodic task but, Ironic will
> correct the state eventually*.
> 
> * See the "sync_power_state_interval" configuration option, it defaults to
> 60 seconds.

Well, that's why I wrote "funny", because I did not expect this behavior. I totally agree with you and I was starting form a clean situation in which the two sync_power_state_interval of Ironic and Nova were set like this:

$ sudo grep -r sync_power_state_interval /etc/
/etc/nova/nova.conf:#sync_power_state_interval=600
/etc/ironic/ironic.conf:#sync_power_state_interval=60

So each one commented and defaulted. I've done the first instance HA test by hanging up a compute node, doing:

# echo c > /proc/sysrq-trigger

and everything behaved as expected:

1) compute got fenced
2) instances got migrated
3) compute came up again, rejoining the cluster

At this point, after a while the compute disappeared again. Looking into the console it was powered off, but the nova and ironic status was "ACTIVE" and "power on".
I tried to force nova by doing a stop and then a start of that compute node instance from the undercloud. Node came up again but after a while got power offed, not by the cluster, since no fencing/stonith operation was made.
So thinking about this bug, I corrected just Nova option sync_power_state_interval to -1, restarted nova services, turned up the node again (by doing nova stop and nova start) and then the node stayed up. Further tests proved that that option made the trick, since everything worked as expected.

Hope things looks more clear now, even anyway strange.

The default setting for nova power sync is set to -1 (meaning disabled), as suggested by Raoul, see comments 39 and 41

You can find the parameter in /etc/nova/nova.conf
sync_power_state_interval=-1

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0637.html