Bug 2039982

+++ This bug was initially created as a clone of Bug #1506372 +++

Proposal summary: We should consider whether pacemaker's handling and reporting of probe return-codes is ideal, or could be improved - such as by reporting OCF_ERR_INSTALLED and OCF_ERR_CONFIGURED less concerningly, or by having that trigger a ban on nodes that gets reported clearly not as an error but as a resource-state condition.  

Background: We have a customer that had configured their cluster with only some nodes having access to certain storage devices that were to be managed by an LVM resource.  Even though they used a constraint to ban nodes that didn't have access, they were getting probe-errors on those nodes and it was concerning their administrators.  We informed them about resource-discovery and this mostly addresses their initial concern.

However, that sparked a deeper discussion about how resource-agents should indicate in their return codes their discoveries about a node being unable to run that resource, and have pacemaker report it less...aggressively?  scarily?  

pacemaker-explained even says:

  "However, there are two situations where disabling resource discovery is a good idea: when a service is not installed on a node, discovery might return an error (properly written OCF agents will not, so this is usually only seen with other agent types);"
  http://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html-single/Pacemaker_Explained/index.html

However, it seems that pacemaker doesn't actually treat any error-return-codes gracefully on probe (other than of course OCF_NOT_RUNNING).  I've tested 1 - 6 and all general Failed Action reports in 'pcs status' / 'crm_mon' output.  

It seems like there is an opportunity for improvement here, to allow probe returns to influence resource placement, and perhaps have those detected conditions reported in a way that is not so intimidating, and perhaps also removes the need for resource-discovery=no constraints to avoid errors.  

  

Version-Release number of selected component (if applicable): pacemaker-1.1.16-12.el7


How reproducible: Not difficult


Steps to Reproduce: 
1. Configure a resource such as LVM or a custom agent that can be setup with some nodes unprepared to host that resource.

2. Cleanup resource, or just look at initial probe results in 'pcs status'


Actual results: 'pcs status' shows errors  that the administrator may take as problems in the cluster, even though they knew that a resource couldn't run in a particular location. 


Expected results: Agents are able to return from a probe in a specific way that indicates to pacemaker whether that node is eligible to run the resource, and pacemaker reports that condition in a useful way in its outputs. 


Additional info:

--- Additional comment from John Ruemker on 2017-10-26 13:05:01 UTC ---

(In reply to John Ruemker from comment #0)
> Proposal summary: We should consider whether pacemaker's handling and
> reporting of probe return-codes is ideal, or could be improved - such as by
> reporting OCF_ERR_INSTALLED and OCF_ERR_CONFIGURED less concerningly, or by
> having that trigger a ban on nodes that gets reported clearly not as an
> error but as a resource-state condition.  
> 

What I was picturing with that suggestion was something like this in crm_mon / pcs status reflecting that we've banned a resource on a node due to its probe result:

 Clone Set: jrummy1-clone [jrummy1]
     Started: [ rhel7-node1.example.com rhel7-node2.example.com rhel7-node3.example.com ]
     Stopped - Not Configured: [ rhel7-node4.example.com ]
     
There could be better terminology to use here, but my point is that we could report this just as a condition of how the resource runs - being banned or stopped on the affected nodes - rather than claiming it as an error.  

<snip>

--- Additional comment from Ken Gaillot on 2022-01-12 18:34:48 UTC ---

Implemented upstream by commits f2e5189~1..ccff6eb

QA: To test, create a ocf:heartbeat:named resource without installing named on at least one node. Before the change, pcs status will show a failed resource action for the probe on any node without named. After, there should be no failed action, but the cluster should not attempt to start named on that node, even if a location constraint prefers it. (Any resource agent without a workaround for probes would work; named just happens to be one.)