Bug 1017827

Summary: Performance regression in cache passivation
Product: [JBoss] JBoss Enterprise Portal Platform 6 Reporter: Dominik Pospisil <dpospisi>
Component: PerformanceAssignee: mmurray
Status: NEW --- QA Contact: Tomas Kyjovsky <tkyjovsk>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: bdawidow, epp-bugs, jpallich, theute
Target Milestone: ER01Flags: jmorgan: needinfo? (dpospisi)
Target Release: 6.1.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: 6_2 Triage JCR_Upgrade
Fixed In Version: Doc Type: Known Issue
Doc Text:
The performance of cache passivation is much lower in EAP/JPP 6.1 Infinispan compared to 6.0. The regression was introduced as a side effect of passivation issues fix. To work-around the issue, disable cache passivation. In the /standalone/configuration/standalone-ha.xml file, comment out the &lt;file-store/&gt; elements in the following block: [source,XML] ---- <cache-container name="web" aliases="standard-session-cache" default-cache="repl" module="org.jboss.as.clustering.web.infinispan"> <transport lock-timeout="60000"/> <replicated-cache name="repl" mode="ASYNC" batching="true"> <!-- <file-store/> --> </replicated-cache> <replicated-cache name="sso" mode="SYNC" batching="true"/> <distributed-cache name="dist" mode="ASYNC" batching="true" l1-lifespan="0"> <!-- <file-store/> --> </distributed-cache> </cache-container> ---- This might have a negative impact on memory consumption as the whole cache is stored in-memory.
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 959630    
Bug Blocks:    

Description Dominik Pospisil 2013-10-10 15:03:03 UTC
Duplicating EAP issue to track this within Portal.

The performance of cache passivation is much lower in EAP/JPP 6.1 Infinispan compared to 6.0. The regression was introduced as a side effect of passivation issues fix.

Workaroud:

Disable cache passivation. This might have negative impact on memory consumption as the whole cache is stored in-memory.

Comment 1 Dominik Pospisil 2013-10-10 19:13:21 UTC
The issue affects EAP's internal caches (not JBossCache based portal caches) - mainly web session and SFSB caches.

To disable the passivation user needs to remove <file-store/> element from cahce config in the server profile configuration.

E.g. to disable web session cache passivation the following fragment of standalone/configuration/standalone-ha.xml

<cache-container name="web" aliases="standard-session-cache" default-cache="repl" module="org.jboss.as.clustering.web.infinispan">
                <transport lock-timeout="60000"/>
                <replicated-cache name="repl" mode="ASYNC" batching="true">
                    <file-store/>
                </replicated-cache>
                <replicated-cache name="sso" mode="SYNC" batching="true"/>
                <distributed-cache name="dist" mode="ASYNC" batching="true" l1-lifespan="0">
                    <file-store/>
                </distributed-cache>
</cache-container>

should be changed to:

<cache-container name="web" aliases="standard-session-cache" default-cache="repl" module="org.jboss.as.clustering.web.infinispan">
                <transport lock-timeout="60000"/>
                <replicated-cache name="repl" mode="ASYNC" batching="true">
                <!--    <file-store/> -->
                </replicated-cache>
                <replicated-cache name="sso" mode="SYNC" batching="true"/>
                <distributed-cache name="dist" mode="ASYNC" batching="true" l1-lifespan="0">
                <!--    <file-store/> -->
                </distributed-cache> 
</cache-container>

Comment 3 JBoss JIRA Server 2013-11-11 16:23:07 UTC
Pedro Ruivo <pruivo> updated the status of jira ISPN-3048 to Coding In Progress

Comment 4 JBoss JIRA Server 2013-11-11 16:36:10 UTC
Paul Ferraro <paul.ferraro> made a comment on jira ISPN-3048

It is also essential that the eviction listeners be notified within the context of the transaction

Comment 5 JBoss JIRA Server 2013-11-11 17:15:06 UTC
Pedro Ruivo <pruivo> made a comment on jira ISPN-3048

hi [~pferraro]

I'm not convinced that the eviction needs to be transactional. It happens under the segment lock in the hash map and it is saved to disk before it is removed from the [in memory] map. Also, if a retrieve operation happens at the same time and the entry is not longer in memory, it is going to look at the cache loader. So, looking at the code, I don't see where the retrieve operation can return null but I'm going to try to do some tests around it.

Anyway, do we have any some kind of discussion to turn the eviction transactional? personally, I don't like the idea...

finally, are you sure that you are not hitting this: https://issues.jboss.org/browse/ISPN-3694
Does it happen with passivation disabled?

thanks!

Comment 6 JBoss JIRA Server 2013-11-25 11:23:40 UTC
Dan Berindei <dberinde> made a comment on jira ISPN-3048

[~pferraro] I agree with Pedro, eviction should not be transactional. If you have a test that fails with size-based eviction, please attach it here. Note that by enabling size-based eviction without passivation, you agree that your data can be lost at any time, and if your isolation level is READ_COMMITTED, even ongoing transactions can see the entry as removed.

[~pruivo] I don't think this is related to ISPN-3694, Paul's problem is with size-based eviction, and the problem Will found is only with manual eviction.

Comment 7 JBoss JIRA Server 2013-11-25 15:01:23 UTC
Pedro Ruivo <pruivo> made a comment on jira ISPN-3048

[~dan.berindei] the size-based eviction is performed under the segment lock in BCHM. How can you loose data? With passivation enabled, I understand that, currently, data can be lost.

Comment 8 Jared MORGAN 2013-12-10 22:08:22 UTC
Hey Dominik, was this supposed to be assigned to me initially? 

I see the CCFR has already been completed - I added the XML fragment in Comment#1 to the Release Note text.

Comment 9 Boleslaw Dawidowicz 2014-02-05 13:55:59 UTC
Related BZ has finally been resolved. Needs to be verified after rebase on EAP 6.2 and JCR upgrade