project_key: JBPAPP6 Filing this issue to investigate why we are seeing so many 500 status codes returned to the client on *graceful* server shutdown. Nothing suspicious in the server log. {noformat} 2012/05/13 18:19:50:360 EDT [WARN ][Runner - 1316] SFCORE_LOG - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Invalid response code: 500 Content: > 2012/05/13 18:19:50:360 EDT [WARN ][Runner - 1620] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Invalid response code: 500 Content: > {noformat}
Job of interest http://hudson.qa.jboss.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-http-session-shutdown-repl-async/19/
To put this in numbers, there are 2000 sessions generating concurency level of ~500 requests/second. Upon 4 server shutdowns there are roughly 1000 errors (500). Interestingly, the number of errors increases with duration of the test.
AS7-4817 seems quite related.
Link: Added: This issue relates to AS7-4817
Link: Added: This issue is incorporated by JBPAPP-7577
Link: Added: This issue relates to AS7-4850
Workflow: Removed: GIT Pull Request workflow Added: jira Security: Added: Public Docs QE Status: Added: NEW
Link: Added: This issue is a dependency of JBPAPP-9290
Link: Removed: This issue is incorporated by JBPAPP-7577
Release Notes Docs Status: Added: Documented as Known Issue Release Notes Text: Added: <code>500</code> status codes are sometimes returned to the HTTP client upon server shut-down. This issue is currently under investigation. Affects: Added: Release Notes
Labels: Added: eap601candidate
Labels: Removed: eap601candidate Added: eap601-qe-triage
Encountered in ER2 testing as well: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-http-granular-shutdown-repl-sync/13/console-perf17/
This happens in other cache modes as well, please see the jobs below. Client was getting response code 500 for about one second after a graceful shutdown of one node even in REPL mode. [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-http-session-shutdown-repl-async/36/console-perf17/ [2] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-http-session-shutdown-dist-async/37/console-perf17/ [3] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-http-session-shutdown-repl-sync/75/console-perf17/ Raising priority to blocker.
Approved for EAP 6.0.1
Labels: Removed: eap601-qe-triage
This issue isnt a regression in 6.0.1. It affected 6.0. Do we want to increase the issues being worked on in 6.0.1? Previously I heard we were only allowing regressions to hold things up.
There were a number of clustering issues that were deferred with the expectation that they would be fixed in a subsequent release. I'd like two questions answered before we go ahead and defer it again. 1. Was this moved out with the expectation of it being fixed in 6.0.1? 2. Does this issue make the QE testing extra difficult? If both of these are answered No, then I agree with Jason and we should move it out since it is not a regression and we are in the end cycles of 6.0.1
This is expected. If a request is accepted by the connector and the server is shutdown, the connector will stop - causing a the 500 error. Fixing this requires clean shutdown logic in the web subsystem. Until that day, you can workaround this issue by using mod_cluster.
As this JIRA has been rejected, release notes are not required.
Release Notes Docs Status: Removed: Documented as Known Issue Added: Not Required Affects: Removed: Release Notes
Release Notes Docs Status: Removed: Not Required Release Notes Text: Removed: <code>500</code> status codes are sometimes returned to the HTTP client upon server shut-down. This issue is currently under investigation. Docs QE Status: Removed: NEW