Description of problem: OSPD 7.1 HA deployment with fencing configured and enabled was upgraded to OSPD 7.2. HA | After upgrading HA dpeloyment from OSPD 7.1 to OSPD 7.2 stonith property becomes false instead of true,therefore fencing won't work. Here is hstory output before the upgrade: ----------------------------------------- 5 pcs stonith create ipmilan-overcloud-controller-0 fence_ipmilan pcmk_host_list=overcloud-controller-0 ipaddr=10.35.160.78 login=admin passwd=admin lanplus=1 cipher=1 op monitor interval=60s; 6 pcs constraint location ipmilan-overcloud-controller-0 avoids overcloud-controller-0; 7 pcs stonith create ipmilan-overcloud-controller-1 fence_ipmilan pcmk_host_list=overcloud-controller-1 ipaddr=10.35.160.38 login=admin passwd=admin lanplus=1 cipher=1 op monitor interval=60s; 8 pcs constraint location ipmilan-overcloud-controller-1 avoids overcloud-controller-1; 9 pcs stonith create ipmilan-overcloud-controller-2 fence_ipmilan pcmk_host_list=overcloud-controller-2 ipaddr=10.35.160.18 login=admin passwd=admin lanplus=1 cipher=1 op monitor interval=60s; 10 pcs constraint location ipmilan-overcloud-controller-2 avoids overcloud-controller-2; 11 pcs property set stonith-enabled=true ---------------------------------------------- As you can see stonith-enable was set to true. After upgrade: [root@overcloud-controller-0 heat-admin]# pcs config | grep stonith Resource: ipmilan-overcloud-controller-0 (class=stonith type=fence_ipmilan) Resource: ipmilan-overcloud-controller-1 (class=stonith type=fence_ipmilan) Resource: ipmilan-overcloud-controller-2 (class=stonith type=fence_ipmilan) stonith-enabled: false Work around: Manually set to true stonith-enably property. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Leonid, I looked at the code and here is how it works: If your environment contains EnableFencing set to True, the property will be automatically managed by Puppet, before and after the upgrade. The EnableFencing parameter is set to False by default so if you don't define it in your environment I don't think you'll have it enabled. My questions for you: * Do you have EnableFencing set to True? * Did you enable Fencing manually by running 'pcs' CLI? If yes, Puppet will override the Fencing configuration by disabling it (since it's disabled by default). If no, we have a bug and I'm interested to dig further. Thanks!
I enable fencing manually by running pcs command via CLI in ospd 7.1 adter updating the system to ospd 7.2 fencing was 'false'
Leonid, that's a normal behavior: Puppet is managing whether we want or do not want Fencing enabled in Pacemaker by using EnableFencing parameter in TripleO Heat Templates. So I don't think this is a bug, but rather something you should fix in our environment by setting EnableFencing to True so Puppet will not override it with False when you upgrade.
Hi Since we don't have auto fencing configuration in ospd 7 , we configure fencing manually post deployment, changing EnableFencing to True is not required by any document https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/7/html-single/Director_Installation_and_Usage/index.html#sect-Advanced-Fencing_the_Controller_Nodes We need to solve it for upgrade as part as upgrade process
We need to note this is the update docs.
There's a note added to the upgrade section: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/7/html-single/Director_Installation_and_Usage/index.html#sect-Updating_the_Overcloud_Packages ==Important== If you configured fencing for your Controller nodes, the update process might disable it. When the update process completes, reenable fencing with the following command on one of the Controller nodes: $ sudo pcs property set stonith-enabled=true === mburns, is there anything else required for this BZ? Or can I close it?
Looks good to me