Red Hat Bugzilla – Bug 900483
Stale session data received when using DIST SYNC on jvm kill
Last modified: 2018-03-06 15:35:49 EST
Affects: Release Notes project_key: JBPAPP6 There are few occurrences when client received older data even when using DIST SYNC. e.g. http://hudson.qa.jboss.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-http-session-shutdown-dist-sync/34/console-perf17/ {noformat} ./perf17.log:2012/05/13 17:11:47:416 EDT [DEBUG][Runner - 968] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Session assigned: hAzyrOe0KxA51coCNbSMUsrZ ./perf17.log:2012/05/13 17:11:47:416 EDT [DEBUG][Runner - 968] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - JvmRoute assigned: perf18 ./perf17.log:2012/05/13 17:14:11:542 EDT [INFO ][Runner - 968] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Failover detected, JvmRoute changed. perf18 -> perf21 ./perf17.log:2012/05/13 17:14:15:546 EDT [WARN ][Runner - 968] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 37, received 36, Runner: 968> ./perf17.log:2012/05/13 17:14:15:546 EDT [WARN ][Runner - 968] SFCORE_LOG - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 37, received 36, Runner: 968> ./perf17.log:2012/05/13 17:21:20:224 EDT [INFO ][Runner - 968] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Failover detected, JvmRoute changed. perf21 -> perf19 ./perf17.log:2012/05/13 17:23:01:507 EDT [INFO ][Runner - 968] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Failover detected, JvmRoute changed. perf19 -> perf21 {noformat} No logging about the session (hAzyrOe0KxA51coCNbSMUsrZ) in the logs.
Link: Added: This issue Cloned from AS7-4818
Link: Added: This issue is incorporated by JBPAPP-7577
Workflow: Removed: GIT Pull Request workflow Added: jira Security: Added: Public Docs QE Status: Added: NEW
Labels: Added: eap6_need_triage
This issue has been triaged and decided to not block/prevent/hold the EAP 6 release. To comply with Release Criteria stating that no issues with Critical or Blocker priority setting can be open for the release, this is changed to priority Major.
Link: Added: This issue is a dependency of JBPAPP-9290
Link: Removed: This issue is incorporated by JBPAPP-7577
Release Notes Docs Status: Added: Documented as Known Issue Release Notes Text: Added: During testing, a few cases showed that stale session data was received when a note shut down and DIST SYNC cache mode was used. This issue is still under investigation. Affects: Added: Release Notes
Labels: Removed: eap6_need_triage Added: eap601candidate
Labels: Removed: eap601candidate Added: eap601-qe-triage
Not approved for EAP 6.0.1. If this should be reconsidered, please add the label: eap601-qe-triage
Labels: Removed: eap601-qe-triage
Writer: Added: mistysj
Release Notes Text: Removed: During testing, a few cases showed that stale session data was received when a note shut down and DIST SYNC cache mode was used. This issue is still under investigation. Added: During testing, a few cases showed that stale session data was received when a note shut down and DIST SYNC or DIST ASYNC cache mode was used. This issue is still under investigation.
Release Notes Text: Removed: During testing, a few cases showed that stale session data was received when a note shut down and DIST SYNC or DIST ASYNC cache mode was used. This issue is still under investigation. Added: During testing, a few cases showed that stale session data was received when a node shut down and `DIST SYNC` or `DIST ASYNC` cache mode was used. This issue is still under investigation.
Release Notes Docs Status: Removed: Documented as Known Issue Writer: Removed: mistysj Release Notes Text: Removed: During testing, a few cases showed that stale session data was received when a node shut down and `DIST SYNC` or `DIST ASYNC` cache mode was used. This issue is still under investigation. Docs QE Status: Removed: NEW
What is the connection with https://bugzilla.redhat.com/show_bug.cgi?id=900776 ? It seems that these two BZ's are duplicates, descended from duplicate JIRA's: https://issues.jboss.org/browse/AS7-4818 https://issues.jboss.org/browse/AS7-5379 This issue has been observed in EAP 6.1.0.ER1, but all progress has been tracked in 900776. Should this be closed as a duplicate? For now, I have set the required flags (jboss-eap-6.1.0, qa_ack).
*** Bug 900776 has been marked as a duplicate of this bug. ***
You are right, these issues are actually the same. Lets keep this one. Its been a while since I filed these issues. Can you now summarize in more detail what the issue is, how often does it happen, what are the conditions, etc.? Thanks!
I will provide more info soon. Since the duplicate was closed there's no need to request information from paul.ferraro (clearing needinfo?).
Can someone from QA verify this against the upcoming build? I'm anticipating that this issue will be addressed by the Infinispan upgrade in BZ 918562. I'm accordingly setting the status to MODIFIED.
I believe that we are still seeing this with EAP 6.1.0.ER3. Looking at https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-session-shutdown-dist-sync/47/console-perf17 (HUGE file, don't open in the browser), there is a LOT of failures looking like this: Stale session data received. Expected 124, received 0, Runner: 1921 Those are bug 901164 (sessions being lost). However, there are also some failures looking like this: Stale session data received. Expected 167, received 128, Runner: 1367 Those look like this particular issue. Also, the upstream JIRA (AS7-4818) is still open. Therefore, reopening.
This may be due to ISPN-2974.
Tested with ISPN 5.2.6.Final. It indeed looks like this issue was fixed in ISPN 5.2.6.Final, but when testing this issue, another problem was found. See BZ 959753.
Update from EAP 6.1.0.ER8 testing: The issue was partially fixed, the number of stale data was lowered, but did not disappeared completely. See the job here: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-session-shutdown-dist-sync/54/ And the parsed client logs: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-session-shutdown-dist-sync/54/artifact/report/parsed_logs_client/index.html
Can we enumerate what is the drop in percentage of the occurence? Thanks
Still seeing this with EAP 6.1.1.ER7. For example: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbservlet-undeploy-dist-sync/20/ https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-session-jvmkill-dist-sync/47/
This will by addressed by the new web session clustering implementation scheduled for 6.3.
Needs to be revalidated following the Infinispan upgrade.
Still present, but since the Infinispan update it occurs only on jvm kill: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-session-jvmkill-dist-sync/63/console-perf17/ https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbservlet-jvmkill-dist-sync/31/
Still an issue, moving to 6.4.
Updating the summary as per comment https://bugzilla.redhat.com/show_bug.cgi?id=900483#c40.
Should be fixed by 5.2.11.CR1 upgrade 1149197
Wasn't fixed by the Infinispan upgrade in EAP 6.4.0.DR5. Moving back to ASSIGNED.