Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 906012

Summary: "All workers are in error state" with a particular ProxyPass setting
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Michal Karm Babacek <mbabacek>
Component: mod_clusterAssignee: Jean-frederic Clere <jclere>
Status: CLOSED NOTABUG QA Contact: Michal Karm Babacek <mbabacek>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: atangrin, rsvoboda
Target Milestone: DR1   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-19 19:36:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
EWS2: Httpd's error_log with Tomcat6
none
EAP6: Httpd's error_log with EAP6
none
EAP6 configuration
none
Tomcat6 configuration
none
ProxyPass config
none
mod_cluster conf
none
Httpd conf
none
HTTP 503 with EAP6
none
Good and expected HTTP 404 with Tomcat6 none

Description Michal Karm Babacek 2013-01-30 15:50:44 UTC 
Created attachment 690429 [details]
EWS2: Httpd's error_log with Tomcat6

I have a test that runs with Tomcat6 (EWS2 release) and with EAP6.
It goes as follows:

1) deploy clusterbench application [1][2]
2) access /clusterbench/requestinfo [2], remember JSESSIONID, JvmRoute    SUCCESS, HTTP 200
3) access /app/doesntExist with an intention to get HTTP 404

With Tomcat6, I get a proper 404 [see tomcat6_HTTP404.log], however with EAP6, I get HTTP 503 and then "All workers are in error state" [see eap6_HTTP503.log].

I would like to draw your attention to the line 208 in [error_log-httpd-with-ews2.log] and 190 in [error_log-httpd-with-eap6.log] where things start to go wrong.
Note that the httpd and native mod_cluster are not the same, despite the m_c 1.2.3.Final and httpd Apache/2.2.22 versions being equal, all the bits come from their project distributions (EWS 2.0.0 and EAP 6.0.1.ER4.2).

ProxyPass, Mod_cluster and Httpd configurations are the same for both EWS2 (Tomcat6) and EAP6 test runs. See attached files.

What might lay at the bottom of this? :-(


[1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/clusterbench-mod_cluster-mbabacek/
[2] https://github.com/Karm/clusterbench/blob/simplified-and-pure/clusterbench-common/src/main/java/org/jboss/test/clusterbench/common/jvmroute/CommonRequestInfoServlet.java
Comment 1 Michal Karm Babacek 2013-01-30 15:51:59 UTC 
Created attachment 690430 [details]
EAP6: Httpd's error_log with EAP6
Comment 2 Michal Karm Babacek 2013-01-30 15:52:56 UTC 
Created attachment 690432 [details]
EAP6 configuration
Comment 3 Michal Karm Babacek 2013-01-30 15:53:34 UTC 
Created attachment 690433 [details]
Tomcat6 configuration
Comment 4 Michal Karm Babacek 2013-01-30 15:54:01 UTC 
Created attachment 690434 [details]
ProxyPass config
Comment 5 Michal Karm Babacek 2013-01-30 15:54:29 UTC 
Created attachment 690456 [details]
mod_cluster conf
Comment 6 Michal Karm Babacek 2013-01-30 15:55:04 UTC 
Created attachment 690457 [details]
Httpd conf
Comment 7 Michal Karm Babacek 2013-01-30 15:55:46 UTC 
Created attachment 690459 [details]
HTTP 503 with EAP6
Comment 8 Michal Karm Babacek 2013-01-30 15:56:28 UTC 
Created attachment 690461 [details]
Good and expected HTTP 404 with Tomcat6
Comment 9 Michal Karm Babacek 2013-02-01 13:49:10 UTC 
Confirmed with EAP 6.1.0 DR2.
Comment 10 Jean-frederic Clere 2013-02-05 16:19:04 UTC 
the 404 comes from the / in tomcat6 to have the same mod_cluster configuration you need to add ExcludedContexts="/" to the configuration.
Comment 13 Michal Karm Babacek 2013-02-07 11:00:41 UTC 
Jean, THX for comments. I will try it again with a different ExcludedContexts as you suggested. ManagerBalancerName setting should be ok...
Comment 14 Jean-frederic Clere 2013-02-07 14:27:06 UTC 
it may be that problems are coming from the:
ProxyPass / !
Why do you have it?
Comment 15 Michal Karm Babacek 2013-02-12 09:39:32 UTC 
Hmm, well, as to "ProxyPass / !", I didn't want with the attached:
    ProxyPassMatch ^/app/static/ !
    ProxyPass /app balancer://qacluster stickysession=JSESSIONID|jsessionid nofailover=on
    ProxyPass / !
    ProxyPassReverse /app balancer://qacluster
    ProxyPassReverseCookieDomain / /app/
    ProxyPassReverseCookiePath / /app/
    ProxyPreserveHost on

get HTTP 200 on app/doesntExist whilst I should be getting (IMHO) HTTP 404 (works with Tomcat6). Nevertheless, with AS7 I get HTTP 503.

Furthermore, with attached
    ProxyPassMatch ^/static/ !
    ProxyPass / balancer://qacluster stickysession=JSESSIONID|jsessionid nofailover=on
    ProxyPassReverse / balancer://qacluster
    ProxyPreserveHost on

I was expecting to get HTTP 404 on doesntExist context (wotks with Tomcat6), but I got HTTP 500 with AS7.
The question is: Can I just modify the test to expect HTTP 500 (503) instead of HTTP 404 while running with AS7, or is there anything more sinister in it?
You may want to take a look at the attached error_log so as to see what broke loose after I accessed the doesntExist context.
Comment 16 Jean-frederic Clere 2013-02-13 07:30:53 UTC 
In EAP6 log ENABLE:
+++
[Wed Jan 30 09:39:40 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=jboss-eap-6.0-1&Alias=default-host%2Clocalhost%2Cexample.com&Context=%2Fclusterbench"
[Wed Jan 30 09:39:50 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=jboss-eap-6.0-2&Alias=default-host%2Clocalhost%2Cexample.com&Context=%2Fclusterbench"
+++
In Tomcat log ENABLE:
+++
[Wed Jan 30 09:35:59 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=tomcat-6-1&Alias=local&Context=%2Fmanager"
[Wed Jan 30 09:35:59 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=tomcat-6-1&Alias=local&Context=%2F"
[Wed Jan 30 09:35:59 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=tomcat-6-1&Alias=local&Context=%2Fdocs"
[Wed Jan 30 09:35:59 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=tomcat-6-1&Alias=local&Context=%2Fexamples"
[Wed Jan 30 09:35:59 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=tomcat-6-1&Alias=local&Context=%2Fclusterbench"
[Wed Jan 30 09:35:59 2013] [debug] mod_manager.c(2598): manager_handler ENABLE-APP (/) processing: "JVMRoute=tomcat-6-1&Alias=local&Context=%2Fhost-manager"
+++

So you are testing different things.
Comment 17 Jean-frederic Clere 2013-02-13 07:32:32 UTC 
please read comment #10
Comment 18 Michal Karm Babacek 2013-02-19 19:36:28 UTC 
I am closing this as NOTABUG.
Jean was right all along. Test suite was updated accordingly both for Tomcat and for AS7.

THX