Bug 1032006 - running gadget server with moderate background activity breaks RTGov
running gadget server with moderate background activity breaks RTGov
Status: CLOSED CURRENTRELEASE
Product: JBoss Fuse Service Works 6
Classification: JBoss
Component: RT Governance (Show other bugs)
6.0.0 GA
Unspecified Unspecified
unspecified Severity unspecified
: ER7
: 6.0.0
Assigned To: Steve Johnson
Andrej Vano
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-19 06:09 EST by Alan Santos
Modified: 2015-02-05 20:19 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1032008 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
asantos: needinfo-


Attachments (Terms of Use)
log file (69.02 KB, text/plain)
2013-11-19 13:50 EST, Alan Santos
no flags Details

  None (edit)
Description Alan Santos 2013-11-19 06:09:16 EST
Description of problem:



Version-Release number of selected component (if applicable):
beta.

How reproducible:
every time

Steps to Reproduce:
1. deploy ordermgmt-app
2. deploy ordermgt-ip
3. open gadget server and add 4 gadgets
4. execute mvn exec:java -Dreq=order1 -Dcount=5000


Actual results:
Errors in log, show here: http://pastebin.com/HjjRSMeu 
widgets do not update


Expected results:
no errors.


Additional info:
Comment 1 Alan Santos 2013-11-19 06:09:58 EST
If the gadget server is not running the errors do not present themselves
Comment 3 kconner 2013-11-19 12:06:42 EST
Gary and I have been unable to reproduce this.  In both instances we ran the tests to completion without seeing any problems.

Can we have the server.log associated with this issue please?  Is there anything else missing in the report?
Comment 4 Alan Santos 2013-11-19 13:50:25 EST
Created attachment 826241 [details]
log file
Comment 5 Gary Brown 2013-11-20 04:52:10 EST
Could you provide details of your environment, fsw version, jdk, os versions, etc. - Kev is using osx aswell so would be good to isolate the differences.
Comment 6 Gary Brown 2013-11-20 05:02:28 EST
Could you also try experimenting with the datasource pool size:

            <pool>
                <min-pool-size>10</min-pool-size>
                <max-pool-size>20</max-pool-size>
                <prefill>true</prefill>
            </pool>

not sure what would be a good max value, but worth trying a few values to see whether there is some sort of leak (i.e. it will eventually fail regardless of value), or whether the problem is related to concurrency being higher than the default number of connections permitted?
Comment 7 Jiri Pechanec 2013-11-20 08:03:45 EST
There are two issues the arises from this report

1) The calltrace widget has problems when there are too many (very big) number of business tx for a single id

2) There is suboptimal query that causes to significantly slow down server when theere are a lot activities

select activityty0_.unitId as unitId2_0_, activityty0_.unitIndex as unitInde3_0_, activityty0_.principal as principa4_0_, activityty0_.tstamp as tstamp5_0_, activityty0_.customType as customTy6_0_, activityty0_.logLevel as logLevel7_0_, activityty0_.message as message8_0_, activityty0_.instanceId as instance9_0_, activityty0_.processType as process10_0_, activityty0_.status as status11_0_, activityty0_.version as version12_0_, activityty0_.variableName as variabl13_0_, activityty0_.variableType as variabl14_0_, activityty0_.variableValue as variabl15_0_, activityty0_.content as content16_0_, activityty0_.messageType as message17_0_, activityty0_.destination as destina18_0_, activityty0_.fault as fault19_0_, activityty0_.interface as interfa20_0_, activityty0_.operation as operati21_0_, activityty0_.serviceType as service22_0_, activityty0_.replyToId as replyTo23_0_, activityty0_.activityType as activity1_0_ from RTGOV_ACTIVITIES activityty0_ inner join RTGOV_ACTIVITY_CONTEXT context1_ on activityty0_.unitId=context1_.unitId and activityty0_.unitIndex=context1_.unitIndex where context1_.value='ID-jpechane-60446-1384949130914-0-540028' and context1_.contextType='Message'; - causes full table scan

If an index is added then it works flawlessly
create index testidx on RTGOV_ACTIVITY_CONTEXT (value,contextType);

Unfortunately there is not much that could be done now with item 1)
Comment 8 Gary Brown 2013-11-20 08:07:43 EST
Thanks Jiri for finding this.

As a longer term enhancement, I think we can add some configurable property to restrict the size of a call trace.

Assigning to productization to apply the index in (2).
Comment 9 Steve Johnson 2013-11-21 12:10:19 EST
Added index for each supported database to ensure where clause of sql statement uses index instead of full table scan.
Comment 10 Andrej Vano 2014-01-15 04:51:14 EST
Hello,

the index is present in the database and the server is not slow with a large number of activities (with different IDs) on CR1. I will open a new BZ for the (1) to not lose the track of it.

Note You need to log in before you can comment on or make changes to this bug.