Bug 979958 - Uploding artifacts to s-ramp takes unreasonably long time
Summary: Uploding artifacts to s-ramp takes unreasonably long time
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Fuse Service Works 6
Classification: JBoss
Component: DT Governance
Version: 6.0.0 GA
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ER4
: ---
Assignee: Thomas Hauser
QA Contact: Martin Vecera
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-07-01 08:23 UTC by Martin Vecera
Modified: 2015-08-31 21:23 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:


Attachments (Terms of Use)
jProfiler snapshot (15.77 MB, application/x-gzip)
2013-07-02 13:57 UTC, Martin Vecera
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker SRAMP-209 0 Major Closed Switch ModeShape to DB persisted (not file) for performance reasons 2015-09-01 17:54:04 UTC

Description Martin Vecera 2013-07-01 08:23:49 UTC
I am running SOA 6 DR6 with s-ramp-server on a powerful machine (Intel Xeon, 12GB RAM), however uploading artifacts from a client like in quickstarts/overlord/s-ramp/s-ramp-demos-simple-client takes unreasonably long time. For example a 5KB artifact takes almost a minute to get uploaded. Server log-in and querying is fine and quick. Just the upload is very slow. It should take at most a couple of seconds.

Comment 1 Martin Vecera 2013-07-01 08:25:01 UTC
In case you could not reproduce, I can run with jProfiler and provide more input. But I already experienced that on 3 different machines. So I suppose this is not isolated.

Comment 2 Kurt Stam 2013-07-01 17:19:44 UTC
hmm not sure what happened to my first comment; trying again: On upload we extract and derive information. So it's actually doing a lot of work. If you can hook up the profiler to see that this is what's going that'd be great. We may be able to do this async, but then it's hard to get errors back to the user.

--Kurt

Comment 3 Len DiMaggio 2013-07-02 13:18:38 UTC
Martin - can you attach the exact artifact (5KB jar file?) that you used?

Comment 4 Martin Vecera 2013-07-02 13:57:12 UTC
Created attachment 767731 [details]
jProfiler snapshot

I have just uploaded a snapshot from jProfiler. Seems like NIO usage in Infinispan is the culprit...

Comment 5 Martin Vecera 2013-07-02 13:58:23 UTC
Len, I used the s-ramp quickstarts - s-ramp-demos-query and s-ramp-demos-simple-client

Comment 6 Kurt Stam 2013-07-02 14:56:49 UTC
FYI This example creates over a 100 artifacts during the upload, for each  ModeShape persists a derived artifact. Not saying we don't have a performance issue, but it's not a "one file upload".

Comment 7 Kurt Stam 2013-07-02 22:02:41 UTC
Horia and Randall offered the following suggestions:

1. Config change: Try a JDBCCache rather then a FileCache: JDBC+H2. This should boost performance on writes. BTW What OS was the Profiler run on?

2. Change code - fewer session.save(), but rather buffer it for a bit. 

3. Config change - async Cache, but this is not without risk.

I think 1 would be an easy thing to try. We used to ship with Berkly DB for this reason but the license was not favorable. Can we (you) try H2?

--Kurt

Comment 8 Martin Vecera 2013-07-03 10:04:58 UTC
1. I created a new H2 datasource and configured the s-ramp cache like below. The defaul isolation level (READ_COMMITED) leads to timeout, so I used NONE. 
I also noticed that there is a missing module dependency - org.infinispan must depend on org.infinispan.cachestore.jdbc. This will be solved with a different BZ.
The OS is Linux x86_64.
                <datasource jndi-name="java:jboss/datasource/OverlordDTGov" pool-name="OverlordDTGov" enabled="true" use-java-context="true">
                    <connection-url>jdbc:h2:${jboss.server.data.dir}${/}h2${/}dtgov</connection-url>
                    <driver>h2</driver>
                    <security>
                        <user-name>sa</user-name>
                    </security>
                </datasource>

           <cache-container name="modeshape">
                <local-cache name="sramp">
                    <transaction mode="NON_XA"/>
                    <locking isolation="NONE" />
                    <mixed-keyed-jdbc-store datasource="java:jboss/datasource/OverlordDTGov" />
                    <!--file-store relative-to="jboss.server.data.dir" path="modeshape/store/sramp" passivation="false" purge="false"/-->
                </local-cache>
            </cache-container>

The user experience with this change is much better. The upload takes significantly less time. With FileCache it even took a very long time to delete the data on the file system (using ext4 with noatime).

Comment 9 Martin Vecera 2013-07-03 10:06:13 UTC
Also please note that I do not have any deep experiences with Infinispan, so I might have used sub-optimal configuration. Maybe string-keyed-jdbc-store would work and could provide better results...

Comment 10 Eric Wittmann 2013-08-15 13:33:53 UTC
S-RAMP has been configured as suggested, both in community and in the product branch.  Hopefully this takes care of the problem!  See the connected JIRA for more information.

Comment 11 Stefan Bunciak 2013-09-10 11:58:42 UTC
There is no specific srampDS datasource (maybe waiting for the DB schemat tool?)

The cache-container is definitely not updated, after installing FSW+DTGov 6.0.0.ER2, the standalone.xml contains:
<cache-container name="modeshape">
                <local-cache name="sramp">
                    <transaction mode="NON_XA"/>
                    <file-store relative-to="jboss.server.data.dir" path="modeshape/store/sramp" passivation="false" purge="false"/>
                </local-cache>
            </cache-container>

Which means https://issues.jboss.org/browse/SRAMP-209 hasn't made it to S-RAMP 6.0.0.ER2.

Comment 12 Stefan Bunciak 2013-09-24 11:49:11 UTC
This still hasn't been fixed. No sramp datasource and modeshape cache-conatiner still configured to use file-store instead of DB after full installation of FSW 6.0.0.ER3.

Comment 14 Stefan Bunciak 2013-10-02 12:18:22 UTC
I've installed FSW 6.0.0.ER4 (using the installer). The configuration/standalone.xml still doesn't contain srampDS neither is sramp cache configured to use jdbc-store instead of file-store.

Do I miss something or why is it still not fixed?

Comment 15 Thomas Hauser 2013-10-02 18:22:31 UTC
These changes were somehow missed. I have made the required changes in the following commit: 

http://git.app.eng.bos.redhat.com/?p=ip.git;a=commit;h=94f54fc44667adaf938cbe360d1e84aa2e8b12b8

Comment 17 Stefan Bunciak 2013-10-07 09:28:25 UTC
Verified in FSW 6.0.0.ER4


Note You need to log in before you can comment on or make changes to this bug.