Description of problem: When loading/updating a fact model (see related bug), the process of updating the model is very slow, to the point that it inhibits the usability of the product. The load time is directly correlated to the number of dependencies, but even a minimal model with slf4j-api, jodatime and junit can result in ~8 second load times for a rule asset. More Logs indicate that this class is to blame: https://github.com/droolsjbpm/kie-wb-common/blob/master/kie-wb-common-services/kie-wb-common-datamodel/kie-wb-common-datamodel-backend/src/main/java/org/kie/workbench/common/services/datamodel/backend/server/cache/LRUDataModelOracleCache.java Version-Release number of selected component (if applicable): How reproducible: Everytime. Steps to Reproduce: 1. Create a project in Business Central with joda-time, sl4j-api and junit as dependencies. 2. Create a guided rule asset. 3. Sit and wait Actual results: Wait time is unreasonable, particularly for a non-technical audience Expected results: Wait time is <2 seconds. Perhaps consider a simplified classpath scanning processing using a magic file for packages a la camel type converters http://camel.apache.org/type-converter.html. This would have the added benefit of eliminating non-fact related class from the fact drop down lists. Additional info:
I tried this with a new project containing the example dependencies with 6.2.0.CR1 (Community, i.e. 6.1 Product). A new Guided Rule *initial* load time (i.e. before LRUDataModelOracleCache has been populated) was ~3s. Subsequent new Guided Rule load times were <1s. Is this not acceptable? Please give the timings you experienced (with 6.2.0.CR1) as there have been some improvements to how the Data Model affected client-side performance. Most notably: https://bugzilla.redhat.com/show_bug.cgi?id=1106469 https://bugzilla.redhat.com/show_bug.cgi?id=1014071 Please also clarify "This would have the added benefit of eliminating non-fact related class from the fact drop down lists"? If a class is on the Project's classpath it will be available for authoring (even if, for example, the User doesn't want some of the classes to be available - e.g. JUnit classes).
I'm note sure I follow the timing scenario you describe. The scenario I'm interested in is from the time an asset is clicked to load until the time it is presented and fully ready to edit. This is what an end user would see. I can give sample load time and log Friday from a project en route to production. As for the drop down item, it is quite common for projects to require dependencies that have nothing to do with rule authoring (e.g. JUnit), even if care is taken to limit the fact model's dependencies (e.g. using xml mapping for persistence instead of annotations). In a guided rule editor scenario, where the target user is non-technical, this shear number of items can be confusing. My suggestion was simply that a targeted scan of the classpath could have the effect of speeding the loading of the cache and eliminating the confusion caused by unnecessary facts in one fell swoop.
(In reply to Justin Holmes from comment #5) > I'm note sure I follow the timing scenario you describe. The scenario I'm > interested in is from the time an asset is clicked to load until the time it > is presented and fully ready to edit. This is what an end user would see. I > can give sample load time and log Friday from a project en route to > production. > > As for the drop down item, it is quite common for projects to require > dependencies that have nothing to do with rule authoring (e.g. JUnit), even > if care is taken to limit the fact model's dependencies (e.g. using xml > mapping for persistence instead of annotations). In a guided rule editor > scenario, where the target user is non-technical, this shear number of items > can be confusing. > > My suggestion was simply that a targeted scan of the classpath could have > the effect of speeding the loading of the cache and eliminating the > confusion caused by unnecessary facts in one fell swoop. Hi Justin, The time taken from clicking on a rule to it appearing in the editor depends on whether the LRUDataModelOracleCache has been populated for the package in which the rule resides. If it has not been cached (or has been invalidated, when for example a Java class is added/deleted/changed or the pom.xml changed etc - anything that can affect the classes available for authoring) then we scan the Project and *all* of its dependencies for classes and build meta-data to aid the client-side rule authoring. This sequence takes longer than if the cache is already populated, in which case we just retrieve the cached object. This aside, how would you envisage we filter the classes available in the Project? What constitutes "unnecessary facts"? Are you advocating use of a meta-data file (e.g. JAR: META-INF/services/org/apache/camel/TypeConverter as mentioned in the Apache Camel Type Converters documentation?). This would surely have to be the exception rather than the norm as I suspect most users would not want to configure additional files to use the workbench. We could implement a mechanism based on a meta-file to restrict which packages have classes shown in the UI however the underlying discovery mechanism uses KIE's Maven artifact/class resolution from a pom.xml. We could investigate changing Maven to use magic marker files too.. but this is probably beyond the scope of what we'd like to do. So at best we could limit the classes LRUDataModelOracleCache processes. Would the meta-data be a white list of black list? Probably, if we implement as a black-list and have it empty by default most users will be ignorant to its existence and more advanced users could configure if necessary. Thanks, Mike
Hey Mike, Sorry for slow response
Hey Mike, Sorry for slow response (and duplicate post). As you can see, I've added links to a support case and a jboss.org issue. It looks like we are suffering from this self referential issue slowing things down dramatically (GUVNOR-2116). RE: meta-data file - yes I am suggesting the exact same thing as the meta data file used by camel for DataType. Only difference being that it would not be required, simply it would override behavior of the classloading if present. I think this is whitelist and definitely for advanced users, especially with complex domains. So once again, this was a suggestion for a potential resolution to assets loading slowly. It's possible that GUVNOR-2116 is sufficient. - Justin
I've added support for a "Package Name White List". The file, imaginatively called "package-names-white-list", should be in the Project root (along with "pom.xml" and "project.imports"). I chose this location over META-INF for consistency as it's where we already store "project.imports". The file is automatically created for new Projects, but if it is missing, or empty all classes in the Maven classpath are available for authoring (i.e. currency behaviour). If the file is not empty it can contain a list of package names (wildcards can be used according to Ant FileSet syntax - https://ant.apache.org/manual/Types/fileset.html).
The "package-names-white-list" file can be edited using the Administration Perspective (it opens in a text editor) or externally, by cloning the repository and pushing commits back.
Just to note, GUVNOR-2116 was fixed by GUVNOR-2140. ============= (master) https://github.com/droolsjbpm/kie-wb-common/commit/b3d55cba88a363dbc2f1b7c3dd5464a63e18a798 https://github.com/droolsjbpm/kie-wb-common/commit/fe0d9e6e7648db814190387da6d7c89860306e89 (6.2.x) https://github.com/droolsjbpm/kie-wb-common/commit/de7bdb49bfe138871ee351eaa52a4e13b475445b https://github.com/droolsjbpm/kie-wb-common/commit/f611795efa37423dace3bcb29cdcb33c2800f95a ============= Btw, now this BZ contains to 2 issues: 1) "Package Name White List" (https://bugzilla.redhat.com/show_bug.cgi?id=1155371#c9) 2) GUVNOR-2116/GUVNOR-2140 "Recursion in ClassFactBuilder leads to StackOverflow when loading in assets into project model cache" (https://issues.jboss.org/browse/GUVNOR-2140) To avoid confusion, GSS will file another BZ for backporting GUVNOR-2140 to BRMS 6.0.x (e.g. rollup patch #2 for BRMS 6.0.3) if required.
According to Toshiya (comments #12), this issue is divided into two issues: 1) a new feature: "Package Name White List" - I verified that it work without problem - I have asserted these useages: - empty white list: I can import all facts from the project and its dependensies into Guided rule - white list contains: -> org.jbpm - I can import just facts from org.jbpm package into Guided rule -> org.jbpm.* - I can import just facts which names matche org.jbpm.anypackage.SomeFact into Guided rule -> org.jbpm.** - I can import all facts which names start with org.jbpm into Guided rule -> combinations of rules. 2) performance problem: Time of loading a guided rule which is located in project with dependencies (joda-time, slf4j-api, junit) - I verified that 6.1.0 ER3 is quicker than 6.0.3 GA
I have created an documentation issue for a white list. Please write there if it may affect users of the engine.
Hi Sir, Can you please give an example, how to specify more than one patterns. I tried giving as below but its not classes are not imported: com.jcms.db.entity.** com.aml.**(In reply to Sona Mala from comment #15) > According to Toshiya (comments #12), this issue is divided into two issues: > > 1) a new feature: "Package Name White List" > - I verified that it work without problem > - I have asserted these useages: > - empty white list: I can import all facts from the project and > its dependensies into Guided rule > - white list contains: > -> org.jbpm - I can import just facts from org.jbpm package > into Guided rule > -> org.jbpm.* - I can import just facts which names matche > org.jbpm.anypackage.SomeFact into Guided rule > -> org.jbpm.** - I can import all facts which names start > with org.jbpm into Guided rule > -> combinations of rules. > > 2) performance problem: Time of loading a guided rule which is located in > project with dependencies (joda-time, slf4j-api, junit) > - I verified that 6.1.0 ER3 is quicker than 6.0.3 GA Hi Sir, Can you please give an example, how to specify more than one patterns. I tried giving as below but its not classes are not imported: com.jcms.db.entity.** com.aml.**
(In reply to Raza from comment #21) > Hi Sir, > > Can you please give an example, how to specify more than one patterns. I > tried giving as below but its not classes are not imported: > > com.jcms.db.entity.** > com.aml.**(In reply to Sona Mala from comment #15) Are you running the product on a Windows server? If that is the case, you might have hit bug 1205180.