Bug 979958 - Uploding artifacts to s-ramp takes unreasonably long time
Uploding artifacts to s-ramp takes unreasonably long time
Status: CLOSED CURRENTRELEASE
Product: JBoss Fuse Service Works 6
Classification: JBoss
Component: DT Governance (Show other bugs)
6.0.0 GA
Unspecified Unspecified
urgent Severity urgent
: ER4
: ---
Assigned To: Thomas Hauser
Martin Vecera
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-01 04:23 EDT by Martin Vecera
Modified: 2015-08-31 17:23 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
jProfiler snapshot (15.77 MB, application/x-gzip)
2013-07-02 09:57 EDT, Martin Vecera
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker SRAMP-209 Major Closed Switch ModeShape to DB persisted (not file) for performance reasons 2015-09-01 13:54:04 EDT

  None (edit)
Description Martin Vecera 2013-07-01 04:23:49 EDT
I am running SOA 6 DR6 with s-ramp-server on a powerful machine (Intel Xeon, 12GB RAM), however uploading artifacts from a client like in quickstarts/overlord/s-ramp/s-ramp-demos-simple-client takes unreasonably long time. For example a 5KB artifact takes almost a minute to get uploaded. Server log-in and querying is fine and quick. Just the upload is very slow. It should take at most a couple of seconds.
Comment 1 Martin Vecera 2013-07-01 04:25:01 EDT
In case you could not reproduce, I can run with jProfiler and provide more input. But I already experienced that on 3 different machines. So I suppose this is not isolated.
Comment 2 Kurt Stam 2013-07-01 13:19:44 EDT
hmm not sure what happened to my first comment; trying again: On upload we extract and derive information. So it's actually doing a lot of work. If you can hook up the profiler to see that this is what's going that'd be great. We may be able to do this async, but then it's hard to get errors back to the user.

--Kurt
Comment 3 Len DiMaggio 2013-07-02 09:18:38 EDT
Martin - can you attach the exact artifact (5KB jar file?) that you used?
Comment 4 Martin Vecera 2013-07-02 09:57:12 EDT
Created attachment 767731 [details]
jProfiler snapshot

I have just uploaded a snapshot from jProfiler. Seems like NIO usage in Infinispan is the culprit...
Comment 5 Martin Vecera 2013-07-02 09:58:23 EDT
Len, I used the s-ramp quickstarts - s-ramp-demos-query and s-ramp-demos-simple-client
Comment 6 Kurt Stam 2013-07-02 10:56:49 EDT
FYI This example creates over a 100 artifacts during the upload, for each  ModeShape persists a derived artifact. Not saying we don't have a performance issue, but it's not a "one file upload".
Comment 7 Kurt Stam 2013-07-02 18:02:41 EDT
Horia and Randall offered the following suggestions:

1. Config change: Try a JDBCCache rather then a FileCache: JDBC+H2. This should boost performance on writes. BTW What OS was the Profiler run on?

2. Change code - fewer session.save(), but rather buffer it for a bit. 

3. Config change - async Cache, but this is not without risk.

I think 1 would be an easy thing to try. We used to ship with Berkly DB for this reason but the license was not favorable. Can we (you) try H2?

--Kurt
Comment 8 Martin Vecera 2013-07-03 06:04:58 EDT
1. I created a new H2 datasource and configured the s-ramp cache like below. The defaul isolation level (READ_COMMITED) leads to timeout, so I used NONE. 
I also noticed that there is a missing module dependency - org.infinispan must depend on org.infinispan.cachestore.jdbc. This will be solved with a different BZ.
The OS is Linux x86_64.
                <datasource jndi-name="java:jboss/datasource/OverlordDTGov" pool-name="OverlordDTGov" enabled="true" use-java-context="true">
                    <connection-url>jdbc:h2:${jboss.server.data.dir}${/}h2${/}dtgov</connection-url>
                    <driver>h2</driver>
                    <security>
                        <user-name>sa</user-name>
                    </security>
                </datasource>

           <cache-container name="modeshape">
                <local-cache name="sramp">
                    <transaction mode="NON_XA"/>
                    <locking isolation="NONE" />
                    <mixed-keyed-jdbc-store datasource="java:jboss/datasource/OverlordDTGov" />
                    <!--file-store relative-to="jboss.server.data.dir" path="modeshape/store/sramp" passivation="false" purge="false"/-->
                </local-cache>
            </cache-container>

The user experience with this change is much better. The upload takes significantly less time. With FileCache it even took a very long time to delete the data on the file system (using ext4 with noatime).
Comment 9 Martin Vecera 2013-07-03 06:06:13 EDT
Also please note that I do not have any deep experiences with Infinispan, so I might have used sub-optimal configuration. Maybe string-keyed-jdbc-store would work and could provide better results...
Comment 10 Eric Wittmann 2013-08-15 09:33:53 EDT
S-RAMP has been configured as suggested, both in community and in the product branch.  Hopefully this takes care of the problem!  See the connected JIRA for more information.
Comment 11 Stefan Bunciak 2013-09-10 07:58:42 EDT
There is no specific srampDS datasource (maybe waiting for the DB schemat tool?)

The cache-container is definitely not updated, after installing FSW+DTGov 6.0.0.ER2, the standalone.xml contains:
<cache-container name="modeshape">
                <local-cache name="sramp">
                    <transaction mode="NON_XA"/>
                    <file-store relative-to="jboss.server.data.dir" path="modeshape/store/sramp" passivation="false" purge="false"/>
                </local-cache>
            </cache-container>

Which means https://issues.jboss.org/browse/SRAMP-209 hasn't made it to S-RAMP 6.0.0.ER2.
Comment 12 Stefan Bunciak 2013-09-24 07:49:11 EDT
This still hasn't been fixed. No sramp datasource and modeshape cache-conatiner still configured to use file-store instead of DB after full installation of FSW 6.0.0.ER3.
Comment 14 Stefan Bunciak 2013-10-02 08:18:22 EDT
I've installed FSW 6.0.0.ER4 (using the installer). The configuration/standalone.xml still doesn't contain srampDS neither is sramp cache configured to use jdbc-store instead of file-store.

Do I miss something or why is it still not fixed?
Comment 15 Thomas Hauser 2013-10-02 14:22:31 EDT
These changes were somehow missed. I have made the required changes in the following commit: 

http://git.app.eng.bos.redhat.com/?p=ip.git;a=commit;h=94f54fc44667adaf938cbe360d1e84aa2e8b12b8
Comment 17 Stefan Bunciak 2013-10-07 05:28:25 EDT
Verified in FSW 6.0.0.ER4

Note You need to log in before you can comment on or make changes to this bug.