Bug 799065 - RFE: apache agent monitor action should not parse content if not asked for it
RFE: apache agent monitor action should not parse content if not asked for it
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: resource-agents (Show other bugs)
7.0
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: David Vossel
Cluster QE
: FutureFeature
Depends On:
Blocks: 915050
  Show dependency treegraph
 
Reported: 2012-03-01 12:00 EST by Jaroslav Kortus
Modified: 2014-06-17 19:55 EDT (History)
5 users (show)

See Also:
Fixed In Version: resource-agents-3.9.5-14.el7
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-06-13 05:57:48 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jaroslav Kortus 2012-03-01 12:00:04 EST
Description of problem:
apache agent checks the status of the service by downloading the content and grepping it for TESTREGEX.

In rgamanger times this was done by comparing running process PID with it's PID file.

I'd like this behaviour to be the default one again, leaving extended checks (like status code, content regexes,...) as an optional check.

This will have several benefits:
 * user migration from rgmanager will be easier
 * service won't be restarted unnecessarily if it serves bad html
 * cluster service status will be by default content independent allowing you to clearly distinguish between cluster configuration mistake or apache's misbehaviour

Version-Release number of selected component (if applicable):
resource-agents-3.9.2-8.el6.x86_64


How reproducible:
always

Steps to Reproduce:
1. echo "Hello Word" > /var/www/html/index.html
2. start apache service via pacemaker
3. see it failing while wget shows you hello world content
  
Actual results:
pacemaker - web content dependency by default

Expected results:
no such dependency, allow optional more in-depth check (for regex, status code,...)

Additional info:
Comment 2 RHEL Product and Program Management 2012-07-10 04:45:21 EDT
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 3 RHEL Product and Program Management 2012-07-10 21:49:02 EDT
This request was erroneously removed from consideration in Red Hat Enterprise Linux 6.4, which is currently under development.  This request will be evaluated for inclusion in Red Hat Enterprise Linux 6.4.
Comment 6 David Vossel 2013-09-04 10:45:24 EDT
(In reply to Jaroslav Kortus from comment #0)
> Description of problem:
> apache agent checks the status of the service by downloading the content and
> grepping it for TESTREGEX.
> 
> In rgamanger times this was done by comparing running process PID with it's
> PID file.
> 
> I'd like this behaviour to be the default one again, leaving extended checks
> (like status code, content regexes,...) as an optional check.
> 
> This will have several benefits:
>  * user migration from rgmanager will be easier

possibly, but the status-url is a better way determining server status rather than just looking at pid files.

>  * service won't be restarted unnecessarily if it serves bad html

This is a configuration problem, I don't believe the service will even start if this is not configured correctly... it shouldn't if it does.

>  * cluster service status will be by default content independent allowing
> you to clearly distinguish between cluster configuration mistake or apache's
> misbehaviour

We check both the pid file and the status url.  The log output provided by the agent should indicate which failed.

> 
> Version-Release number of selected component (if applicable):
> resource-agents-3.9.2-8.el6.x86_64
> 
> 
> How reproducible:
> always
> 
> Steps to Reproduce:
> 1. echo "Hello Word" > /var/www/html/index.html
> 2. start apache service via pacemaker
> 3. see it failing while wget shows you hello world content

I don't understand why the apache service is failing here. Did you set the status-url to index.html for some reason.  The typical deployment sets the status-url field to http://localhost/server-status.  That page is something Apache serves up just to the local machine that displays health/status info.  This url is actually discovered by default as long as you setup the server-status feature correctly in the Apache configuration.  The "Clusters from Scratch" document walks-through the general procedure used to configure and start an apache service using this agent. Usage of the server-status page is even documented in the agent's metadata for the 'statusurl' option.

http://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html-single/Clusters_from_Scratch/index.html#_enable_the_apache_status_url

-- Vossel

> Actual results:
> pacemaker - web content dependency by default
> 
> Expected results:
> no such dependency, allow optional more in-depth check (for regex, status
> code,...)
> 
> Additional info:
Comment 7 Jaroslav Kortus 2013-09-05 06:50:11 EDT
> I don't understand why the apache service is failing here. Did you set the
> status-url to index.html for some reason.  The typical deployment sets the
> status-url field to http://localhost/server-status.  That page is something
> Apache serves up just to the local machine that displays health/status info.
> This url is actually discovered by default as long as you setup the
> server-status feature correctly in the Apache configuration.  The "Clusters
> from Scratch" document walks-through the general procedure used to configure
> and start an apache service using this agent. Usage of the server-status
> page is even documented in the agent's metadata for the 'statusurl' option.
> 
> http://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html-single/
> Clusters_from_Scratch/index.html#_enable_the_apache_status_url

It fails because it parses /'s content. It also looks for some exact html content (regex matching </ *html *>), which is what I did not like.

I had not configured any status url manually (just used what yum install httpd provided) and it still started. Monitor op asks for "/" and no /server-status.

That being said, I don't understand it at all now ;). On one hand you require /server-status, but on the other hand all you are looking for is HTML page ending. If it's not there, "/" is sufficient. This does not make much sense to me. Do we REALLY need /server-status for any valuable info?

What would make much more sense to me would be a test if apache is responding (with whatever content, error msgs, anything really). Just check that it sends some reply on configured URL/port.

This is a bit tricky with current wget/curl implementation as wget returns EC>0 for server error messages (like 404, 401).

So my updated wish is:
1. standard status check checks just apache's ability to serve content on supplied URL ("/" by default), no regexes checked (or empty regex if it's easier to implement). The content of "/" might be anything, even an error page.

2. extended status check is available for users wishing to do more extended checks. This is implemented already, users would have just to specify testregex parameter if they are looking for any particular string or go with OCF_CHECK_LEVEL=10 checks.
Comment 8 David Vossel 2013-09-05 11:15:46 EDT
(In reply to Jaroslav Kortus from comment #7)

> I had not configured any status url manually (just used what yum install
> httpd provided) and it still started. Monitor op asks for "/" and no
> /server-status.
> 
> That being said, I don't understand it at all now ;). On one hand you
> require /server-status, but on the other hand all you are looking for is
> HTML page ending. If it's not there, "/" is sufficient. This does not make
> much sense to me. Do we REALLY need /server-status for any valuable info?
> 
> What would make much more sense to me would be a test if apache is
> responding (with whatever content, error msgs, anything really). Just check
> that it sends some reply on configured URL/port.
> 
> This is a bit tricky with current wget/curl implementation as wget returns
> EC>0 for server error messages (like 404, 401).

I think we'd want to limit the monitor success to return codes in the 2XX range.

> So my updated wish is:
> 1. standard status check checks just apache's ability to serve content on
> supplied URL ("/" by default), no regexes checked (or empty regex if it's
> easier to implement). The content of "/" might be anything, even an error
> page.

I agree, This sounds reasonable.  It is a deviation from the current default behavior, but it might be possible to detect if the server-status feature is enabled and use that over "/" like people have been doing.

There might be some benefit/necessity to the server-status feature that neither of us are aware of though.  I'll present this upstream and see if anyone complains.

-- Vossel
Comment 9 David Vossel 2013-09-10 17:32:24 EDT
A patch related to this issue exists upstream.

https://github.com/davidvossel/resource-agents/commit/834c37752c5d2079fff450c90e05a9c0fce34ac7
Comment 10 David Vossel 2013-09-12 17:17:47 EDT
ready for testing.  The apache agent should no longer require the server-status page to be enabled for default basic request/response monitoring.
Comment 13 michal novacek 2013-11-29 08:24:01 EST
I have verified that ocf:heartbeat:apache from resource agents 3.9.5-20 can be
started even though apache's index.html file does not contain valid html and
/server-status page is unavailable.

virt-134$ rpm -q resource-agents
resource-agents-3.9.5-20.el7.x86_64

virt-134$ # pcs status
Cluster name: STSRHTS32645
Last updated: Fri Nov 29 14:13:04 2013
Last change: Fri Nov 29 14:11:26 2013 via cibadmin on virt-134
Stack: corosync
Current DC: virt-134 (1) - partition with quorum
Version: 1.1.10-20.el7-368c726
3 Nodes configured
2 Resources configured

Online: [ virt-134 virt-135 virt-136 ]

Full list of resources:

 virt-fencing   (stonith:fence_xvm):    Started virt-134 
 webserver      (ocf::heartbeat:apache):        Started virt-134 

PCSD Status:
  virt-134: Offline
  virt-135: Offline
  virt-136: Offline

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: unknown/disabled

virt-134$ # pcs resource show webserver
 Resource: webserver (class=ocf provider=heartbeat type=apache)
  Attributes: configfile=/etc/httpd/conf/httpd.conf 
  Operations: monitor interval=30s (webserver-monitor-interval-30s)

virt-134$ wget --server-response -O - -q http://virt-134
  HTTP/1.1 200 OK
  Date: Fri, 29 Nov 2013 13:22:11 GMT
  Server: Apache/2.4.6 (Red Hat)
  Last-Modified: Fri, 29 Nov 2013 13:10:25 GMT
  ETag: "2b-4ec508da87270"
  Accept-Ranges: bytes
  Content-Length: 43
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: text/html; charset=UTF-8
virt-135.cluster-qe.lab.eng.brq.redhat.com

virt-134$ wget --server-response -q http://virt-134/server-status
  HTTP/1.1 404 Not Found
  Date: Fri, 29 Nov 2013 13:21:41 GMT
  Server: Apache/2.4.6 (Red Hat)
  Content-Length: 211
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: text/html; charset=iso-8859-1
Comment 14 Ludek Smid 2014-06-13 05:57:48 EDT
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Note You need to log in before you can comment on or make changes to this bug.