Bug 110325 - some Ant targets can be parameterized to reduce the size of build.xml generated by ccm-configure.sh
some Ant targets can be parameterized to reduce the size of build.xml generat...
Product: Red Hat Web Application Framework
Classification: Retired
Component: dev environment (Show other bugs)
All Linux
medium Severity high
: ---
: ---
Assigned To: Dennis Gregorovic
Jon Orris
Depends On:
Blocks: 106481 110537
  Show dependency treegraph
Reported: 2003-11-18 10:10 EST by Vadim Nasardinov
Modified: 2007-04-18 12:59 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2004-01-23 10:24:03 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Ant dependency graph for a single project. (22.97 KB, image/png)
2003-12-03 15:18 EST, Vadim Nasardinov
no flags Details
A list of targets in APLAWS build.xml grouped by type (44.75 KB, text/plain)
2003-12-09 11:07 EST, Daniel Berrange
no flags Details
tiny-build.xml (3.64 KB, text/plain)
2003-12-12 12:47 EST, Sebastian Skracic
no flags Details

  None (edit)
Description Vadim Nasardinov 2003-11-18 10:10:58 EST
The build.xml file generated by ccm-configure.sh has a lot of nearly
identical targets that differ only in the application key (which is
the "name" attribute of <ccm:application> element in project.xml).
Some of these targets are long enough to benefit from

Specifically, if my project.xml contains the following:

   <ccm:application name="core"/>
   <ccm:application name="cms"/>
   <ccm:application name="article"/>

Then the corresponding build.xml file will have targets named
clean-tests-core, clean-tests-cms, and clean-tests-article.  These
three are nearly identical.  To illustrate, here are the first two:

 <target depends="init,clean-tests-core-hook" name="clean-tests-core">
         <fileset dir="core">
             <include name="TEST-*.txt"/>
             <include name="TEST-*.xml"/>

 <target depends="init,clean-tests-cms-hook" name="clean-tests-cms">
         <fileset dir="cms">
             <include name="TEST-*.txt"/>
             <include name="TEST-*.xml"/>

These targets can be rewritten as follows:

 <target depends="init,clean-tests-core-hook" name="clean-tests-core">
   <antcall target="parameterized-clean-tests">
     <param name="app.name" value="core"/>

 <target depends="init,clean-tests-cms-hook" name="clean-tests-cms">
   <antcall target="parameterized-clean-tests">
     <param name="app.name" value="cms"/>

 <target name="parameterized-clean-tests">
         <fileset dir="${app.key}">
             <include name="TEST-*.txt"/>
             <include name="TEST-*.xml"/>

In this particular case, the LOC savings are minimal.  However, many
of the targets are longer than the "clean-tests-*" example I picked
and would benefit from parameterization much more.  (On the other
hand, parameterizing shorter targets may actually increase the LOC
count rather than decrease it.)

For a project.xml file with three applications, the LOC savings may be
insignificant.  For a larger project, the difference might be a lot
more noticeable.  I think Dan reported working with build.xml files
that are in excess of 5MB.
Comment 1 Vadim Nasardinov 2003-11-18 10:17:31 EST
This doesn't address the O(n) problem, it just tries to curb it
somewhat.  I don't see a way to make the build.xml size O(1) without
circumventing Ant's dependency mechanism.  Specicifically, I think
Ant insists on having all dependency spelled out statically via
the "depends" attribute of the "target" element.  If there was
a way to specify dependencies dynamically at runtime, then
the size of build.xml could be made O(1).
Comment 2 Daniel Berrange 2003-11-18 10:20:42 EST
Indeed the project.xml for APLAWS containing a huge number of apps:

dan@camden$ grep ccm:application project.xml   | wc -l
dan@camden$ ls -lh build.xml 
-rw-rw-r--    1 dan      ccm-deve     2.1M Nov 10 10:10 build.xml
dan@camden$ time ant init
     [echo] /var/ccm-devel/dev/dan/aplaws-rickshaw/build.xml

Total time: 27 seconds

real    0m28.685s
user    0m31.780s
sys     0m0.480s

I think we should have two goals:

1. Reduce the startup time
2. Reduce the time to do a complete 'ant deploy' when no files have

Point 1. is most urgently needed, since we've got a fixed 30 sec
startup overhead at the mo. Most of that time seems to be parsing &
processsing the build.xml.

Parameterized targets look like a definite candidate for lowering this
overhead. Another improvment may come from using the ant-contrib
package which provides an '<if>' conditional. This would lets us
combine many of the wrapper targets, eg _compile-core & compile-core.


Finally, we should probably audit just what targets we have defined -
we may find that some are completely unneccessary.
Comment 3 Daniel Berrange 2003-11-18 10:23:29 EST
Oh as a final note, we should change the dependencies we encode so
that instead of using the @buildOrder attribute  or 'position()' of
current <ccm:application> tag, it would pull in the actual
dependencies defined in the application.xml file.

For example, the 'xmlfeed' application is listed 34th in the
project.xml. So if I wan't to compile it with the current method, ant
will first  compile the previous 33 applications, however, xmlfeed
only depends on Core & CMS, so the minimal build is actually only 2
Comment 4 Vadim Nasardinov 2003-11-18 10:29:11 EST
The final should probably go under bug 107191.  Let's keep this ticket
reasonably small.
Comment 5 Vadim Nasardinov 2003-11-18 11:38:51 EST
On the face of it, parsing a 5MB file should not take 30 seconds.  So
I thought that maybe Ant takes a lot of time to process target
dependencies.  I set up a large number of (fake) apps like so:

|$ cat /tmp/fakeapps.sh 
|for nn in `seq 50`; do
|    app="fakeapp$nn"
|    ln -s /var/vadim/p4checkout/core-platform/dev $app
|    echo "<ccm:application name=\"$app\" prettyName=\"Fake App\" buildOrder=\"$nn\"/>"

After modifying project.xml and rerunning ccm-configure.sh, the
resulting build.xml file is abut 3MB:

|$ ls -l build.xml 
|-rw-rw-r--    1 vadim    ccm-devel  3058810 Nov 18 11:25 build.xml

The time it takes to parse this file is negligible:

|$ java -classpath /tmp/xp.jar com.jclark.xml.apps.Time build.xml

Running "ant init" produces results very similar to Dan's.

|$ grep -c depends build.xml
|$ time ant init
|Total time: 44 seconds
|real    0m45.270s
|user    0m35.410s
|sys     0m0.380s

Removing all the "depends" attributes from "target" elements changes

|$ perl -i.bak -npe 's/depends="[^"]+"//' build.xml 
|$ grep -c depends build.xml
|$ time ant init
|Total time: 40 seconds
|real    0m41.037s
|user    0m35.230s
|sys     0m0.250s
Comment 6 Daniel Berrange 2003-11-18 11:51:35 EST
I can't find the link just now, but, although build.xml may *look*
like XML, ANT doesn't actually use a standard XML parser - it is hand
crafted. For example it complains bitterly if you add an
xmlns:foo="http://www.example.com" attribute to any tag.

Without using a profiler we can't really say with much confidence
whether it is parsing, or constructing the in memory objects while
parsing that takes the time. optmizeit would likely show us the
hotspot(s) very quickly.....

Comment 7 Vadim Nasardinov 2003-12-03 15:18:07 EST
Created attachment 96319 [details]
Ant dependency graph for a single project.

The diagram was generated with the help of

I set up a dummy project.xml that contained a single application named "p" for
brevity.  I generated build.xml by running ccm-configure.sh and fed it to the
above-mentioned script like so: "$ ant2dot.sh build.xml init".

I removed the "init" node (and all its incident edges) to unclutter the graph.
Comment 8 Vadim Nasardinov 2003-12-03 15:23:23 EST
Whether the Ant's homegrown parser is too slow or constructing
in-memory target objects takes a lot of time, it seems clear that
reducing the size of build.xml would reduce build times.

One thing we can do is write a wrapper script in Python or Perl (or in
pure XSLT for the masochistically inclined) that would trim down
build.xml by removing targets that are not required for the execution
of the requested target.

For example, as shown in attachment 96319 [details], running
"deploy-jar-classes-p" does not require any other target that starts
with "deploy-".  It only requires "build-p" and its prerequisites.
The wrapper script can compute all the required prerequisites and spit
out a temporary file trimmed-build.xml that contains only the necessary
targets.   It can then invoke ant like so:

$ ant deploy-jar-classes-p trimmed-build.xml

instead of the slower

$ ant deploy-jar-classes-p build.xml
Comment 9 Daniel Berrange 2003-12-05 06:52:40 EST
Perhaps we are looking at the problem from totally the wrong angle.
The reason why build.xml is so large is that each application has 30
odd targets & there are 20-30 applications. Now, why do we need to
have separate targets per application ? It is because running a full
'ant deploy' is too slow, that people like to shortcut with 'ant
deploy-myapp'. However, we've added many dependancies between targets
so 'deploy-myapp' practically does the same thing about the full

So if we removed *all* per-application targets the build.xml would be
tiny. If we looked at each target & ensured it had optimal execution
then running 'deploy' would be fast enough that there is no need for
per-app targets.

The only case for a per-app target I see is 'compile-myapp' - useful
when you have written a bunch of code & you are iterating over
compile, view errors, fix, compile, etc cycles. In this use case there
is no need for dependancies on other compile targets & since we're not
deploying there's no risk of an inconsistent webapp.

Incidentally I've profiled 'ant init' and it seems the big overhead is
in constructed ant's in memory representation of the build.xml file.
We could try and optmize ant's  code for this (they use Strings
instead of StringBuffers when processing the character(byte[], int,
int) method of SAX DefaultHandler :-( ). Failing that, the only way to
reduce time is to make build.xml smaller.
Comment 10 Daniel Berrange 2003-12-05 07:11:12 EST
I'm marking this as high since slow performance of ANT is really
impacting on development times for people working on > 10 apps at once.
Comment 11 Sebastian Skracic 2003-12-05 10:24:26 EST
And what about people with > 30 apps?  This whole build thing is on
crack.  I'm inclined to devote several days to write my own build
system -- eventually it'll make my life easier and more productive. 
On my box 'ant init' takes 40 seconds!!!

My suggestion is: scrap everything!  Split the damned things into 10
or more separate build.xml files.  I'm not interested in ant checking
for data model changes when I'M FIXING THE SINGLE TYPO IN JAVA CODE!!!
 I'm not interested in running junit test when MY CODE DOESN'T COMPILE!!!

Or better yet, scrap ant, make/make+ is the way to go.
Comment 12 Richard Li 2003-12-05 13:13:46 EST
One other thought: we could slice the number of apps by 10 if we made
all the content types one app, instead of many. 

You could install the entire CT RPM, and then use the UI to
selectively install the CTs in the UI. The disadvantage is that you
can't incrementally update each of the separate CTs, but I think
that's OK for CTs that are part of product.
Comment 13 Vadim Nasardinov 2003-12-05 13:28:37 EST
> So if we removed *all* per-application targets the build.xml would
> be tiny.
> the big overhead is in constructed ant's in memory representation of
> the build.xml file.  We could try and optmize ant's code for this

That might the best long-term solution (provide we push our Ant patch
upstream).  As a short-term solution, the wrapper script approach
seems better, as it preserves the current behavior intact, while
likely offering a substantial speedup.

> Split the damned thing into 10 or more separate build.xml files.

That's effectively what the wrapper script approach would do.

> I'm not interested in running junit test when MY CODE DOESN'T

"ant compile-core" or some such does not have "runtests" as a
prerequisite.  See compile-p in attachment 96319 [details].

> Or better yet, scrap ant, make/make+ is the way to go.

This has been proposed, too.  I think Rafi toyed with this idea.

> This whole build thing is on crack.

Most targets (except for "larry", "kaboom" and maybe a few others)
were motivated by bugs at some point in the past.  Looking at
attachment 96319 [details], which targets (or dependencies) would you suggest
cutting out?
Comment 14 Sebastian Skracic 2003-12-06 12:50:35 EST
> "ant compile-core" or some such does not have "runtests" as a
> prerequisite.  See compile-p in attachment 96319 [details].

  I know that.  My point is: I spent 99% of my ant time compiling the
thing.  If I run junit tests, then I can accept 40 seconds startup
delay, because a) I'm not running them that often b) the task itself
takes substantially longer then 40 seconds.

  I vote for 2 (worded: TWO) build.xml files.  The first would contain
only compile/deploy targets.  The second one would be everything else,
actually the one we currently have.  The first should be ~50 k in
size, the other one 2MB+ (in case of 30+ applications).
Comment 15 Vadim Nasardinov 2003-12-06 15:31:40 EST
> My point is: I spent 99% of my ant time compiling the thing.

Ah, I see what you mean.  Makes sense.  As long as we are campaigning
to reduce compilation times, I'd like to mention JavaMake again:
Comment 16 Daniel Berrange 2003-12-08 04:37:52 EST
> One other thought: we could slice the number of apps by 10 if we made
> all the content types one app, instead of many.

In short, not an option. We've been through this discussion before. It
is a requirement of APLAWS-II that the content types are separated out.
In any case we would still have 20 applications & 1MB build.xml.
Comment 17 Daniel Berrange 2003-12-09 11:07:05 EST
I've analysed just what targets are being generated in build.xml and
some definite optimizations come to mind. For APLAWS build.xml, there
are 1936 targets in general. Of out these:

 * 148 end in -hook. These are checks to see if a build-hoooks.xml
file exists. Only two applications (core&chat) currently uses this
capability. Rather than having targets that check for presence of
build-hooks.xml per run, we should have an attribute on
application.xml saying whether there is a build-hooks for this app.
The check is then a one-time event & we can remove 148 targets.

 * 224 end in -check. These are checks to see if various top level
directories (eg, pdl, sql, src, web) exist & thus whether or not to
run the corresponding compile/deploy type targets for each type of
file. Again nearly every single app has all these directories all the
time. We should add an attribute to application.xml which developers
can use to explicitly inform us if a particular directory is not used.
We can then remove 224 targets & speed up build time by not doing all
these checks. 

By removing these two items we can combine 'compile-[appname]' and
'_compile-[appname]' into one target.

 * 187 contain 'jar'. These are only used when making binary
distributionms. We can strip these all out & put them in a separate
build-dist.xml. The build scripts can then be updated to pass in '-f
build-dist.xml' when running ant. This will also elminate a large
number of other tasks deploy related tasks

* There a debugtests and runtests. Does anyone ever use debugtests ?
Can we remove it & just let them pass '-Dsomeprop=true' as an option
to 'runtests'.

Doing the first two points alone will simplify build.xml dramatically
& remove a huge number of filesystem checks, without causing any
serious change in functionality or compatability problems. This should
be top priority since it gives us a short term win for very little effort.

Subsequently we should consider the splitting of build.xml for the
build scripts.
Comment 18 Daniel Berrange 2003-12-09 11:07:43 EST
Created attachment 96424 [details]
A list of targets in APLAWS build.xml grouped by type
Comment 19 Dennis Gregorovic 2003-12-09 15:19:36 EST
implemented Dan's suggestion to eliminate unnecessary -hooks targets
Comment 20 Dennis Gregorovic 2003-12-11 16:48:50 EST
change 38711 implements Dan's suggestion to do the directory checks at
ccm-configure.sh time instead of at ant time.  Then @38728 I checked
in updated application.xml files for all of the content types with
their directory info.  

I have compared the current build.xml with the build.xml before 38639.
 In my project.xml I have 21 applications - core, cms, and 19 content
types.  The number of targets has been cut in half and the startup
time reduced by 25%.

# new
04:41:10 dgregor@galileo foo4$ time ant init
Buildfile: build.xml

     [echo] /var/ccm-devel/dev/dgregor/foo4/build.xml

Total time: 9 seconds

real	0m10.272s
user	0m8.710s
sys	0m0.170s
04:41:29 dgregor@galileo foo4$ ant -projecthelp | grep "^ " | wc -l

04:43:36 dgregor@galileo foo4$ time ant init
Buildfile: build.xml

     [echo] /var/ccm-devel/dev/dgregor/foo4/build.xml

Total time: 12 seconds

real	0m12.677s
user	0m10.900s
sys	0m0.280s
04:43:55 dgregor@galileo foo4$ ant -projecthelp | grep "^ " | wc -l
Comment 21 Dennis Gregorovic 2003-12-11 17:31:35 EST
Another idea is that we can turn ant into a daemon.  So, the first
time you run it, it loads build.xml into memory and sets up the
objects, etc.  Then it just sticks around, listening on some port. 
Subsequent calls use the running ant process.  This could definitely
be a cool feature to push upstream.
Comment 22 Vadim Nasardinov 2003-12-11 18:07:26 EST
I thought someone has done that already.  I vaguely remember
Mike Bonnet playing with some hacked up daemon-enabled
version of Ant eons ago.
Comment 23 Sebastian Skracic 2003-12-12 12:47:01 EST
Created attachment 96498 [details]

  Being extremely frustrated with the current situation, I decided to
write my own tiny-build.xml which would take care only of often needed
targets.  I started dropping the targets I don't use often (for them
I can always resort to autogenerated build.xml). however that approach
led me nowhere -- my total ignorance of XSL being one of the obstacles.

  So I took the opposite approach, and started writing tiny-build.xml
with only one target: compile (which I use 90% of the time).  Eventually I
came up with 3 targets: precompile, compile and deploy.

  The idea was to have single build/classes directory where all compiled
classes should go.  All */src directories are included in compile
source path. 

  The problem started when I tried to run some command tool with
'ccm-run'.  It turned out that ccm-run utilizes ccm.classpath file which
gives precedence to classes compiled in <app>/build/classes directories
of individual applications, instead of $CCM_HOME/webapps/WEB-INF/classes,
where the tiny-build.xml (and the original build.xml) deploys compiled

  It stroke me as something illogical -- the consequence being I cannot
use tiny-build.xml to compile classes run by ccm-run.  So I reduced
ccm.classpath to following:


  After having done that, ccm-run was happy with the classes deployed
by tiny-build.xml.

  Here's the quick comparison of autogenerated build.xml and
tiny-build.xml.  The test was run on most frequently used scenario:
single change in single class file (!).

  First, sizes:

[sskracic@tuborg coventry]$ ls -l *build.xml
-rw-rw-r--    1 sskracic ccm-devel  1272555 Dec 10 11:29 build.xml
-r--r--r--    1 sskracic ccm-devel     3726 Dec 12 17:16 tiny-build.xml

  Execution times for single change in single class file;

  build.xml:	 42 seconds
  tiny-build.xml: 6 seconds
Comment 24 Vadim Nasardinov 2003-12-12 13:33:34 EST
> The idea was to have single build/classes directory where all
> compiled classes should go.  All */src directories are included in
> compile source path.

This makes it possible to introduce undesirable dependencies.  With
the above setup, you can make Core depend on CMS, and stuff will
happily compile.  May not be a showstopper, but definitely something
to keep eye an out for.
Comment 25 Dennis Gregorovic 2003-12-12 14:02:22 EST
> So I reduced ccm.classpath to following:
> /var/ccm-devel/web/sskracic/coventry/webapps/WEB-INF/classes
> /var/ccm-devel/web/sskracic/coventry/webapps/WEB-INF/lib
> /usr/share/java

The reason that ccm.classpath in the development environment has that
directories that it has is because the build system separates
'building' from 'deploying', and ccm.classpath is set up to use the
build directories, not the deploy directories.  So, there are actually
two subtle issues here.

1) Should the build system separate building and deploying?  One
benefit to the separation is that you can build (and run tests)
without affecting your running server.  Also, the separation provides
functionality with little drawback as a developer can just as easily
run 'ant deploy' as 'ant build'

2) Should ccm.classpath use the build directories or the deploy
directories?  I could be convinced pretty easily to switch to using
the deploy directories.  What do other people think?

Now, switching gears in thought, I do not like the idea of splitting
build.xml into multiple files for a few reasons.  However, I am in
favor of adding a switch to ccm-configure that would let the user
specify the template file, and I could add a build-template-lite.xsl
to the ccm-devel package.
Comment 26 Vadim Nasardinov 2003-12-15 10:57:20 EST
The following properties appear to be unused:

Comment 27 Dennis Gregorovic 2003-12-23 15:45:15 EST
@39055 I changed build-template.xsl to better use path refs, reducing
reduncancy in classpath declarations.

wc -l build.xml
  36023 build.xml

wc -l build.xml
  17511 build.xml
Comment 28 Vadim Nasardinov 2004-01-06 11:24:19 EST
Dennis, what about the "app.prettyname" property that was introduced
in change 39057?  Where is it supposed to be defined?
Comment 29 Vadim Nasardinov 2004-01-06 11:35:17 EST
I downloaded and installed the latest ccm-devel-2.0.0-2 RPM from
park-street this morning, and here's the list of unused properties
that I get:

$ xsltproc \
 /var/vadim/p4checkout/users/vadim/code/xsl/build-properties.xsl \
 build.xml | \
 /var/vadim/p4checkout/users/vadim/code/xsl/build-properties.pl | sort
Comment 30 Dennis Gregorovic 2004-01-23 10:24:03 EST
I am closing this ticket.  Since the ticket was filed, a lot of
performance improvements have been made to build.xml.  With a test
project of 41 applications, the number of lines in in build.xml is now
1/3 of what it was before, and startup time is 1/4th.

# as of Nov-18-2003
dgregor@galileo project$ wc -l build.xml
  46069 build.xml
dgregor@galileo project$ time ant init > /dev/null
                 real    0m29.828s
user    0m28.940s
sys 0m0.310s

# as of Jan-23-2004
dgregor@galileo project$ wc -l build.xml
  15847 build.xml
dgregor@galileo project$ time ant init > /dev/null
                 real    0m7.927s
user    0m7.140s
sys 0m0.270s

Note You need to log in before you can comment on or make changes to this bug.