Bug 51792 - Optimized byte-code should be provided also
Summary: Optimized byte-code should be provided also
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: 4Suite
Version: 7.3
Hardware: i386
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Trond Eivind Glomsrxd
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2001-08-15 10:01 UTC by Enrico Scholz
Modified: 2008-05-01 15:38 UTC (History)
0 users

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2001-09-20 18:42:47 UTC
Embargoed:


Attachments (Terms of Use)

Description Enrico Scholz 2001-08-15 10:01:13 UTC
4Suite-0.11-2 contains only normal .pyc byte code but node the optimized
.pyo one:

| $ rpm -ql 4Suite | grep pyo
| $

This leads to significant slowdowns when used in an optimized project.

E.g.:

-----------
$ cat test.py
import xml.xslt.Processor
import xml.dom;

dt = xml.dom.DocumentType.DocumentType('foo', None, None, None, None)
dc = xml.dom.Document.Document(dt)
dc.createElement('bar')
proc = xml.xslt.Processor.Processor()
$ python test.py && time python test.py    ## (1)
real    0m4.891s
user    0m0.970s
sys     0m0.110s

$ python -O test.py && time python -O test.py ## (2)
real    0m4.002s
user    0m3.930s
sys     0m0.070s

$ su -
# python -O test.py && time python -O test.py ## (3)
real    0m1.027s
user    0m0.910s
sys     0m0.110s

$ exit 

$ python -O test.py && time python -O test.py ## (4)
real    0m1.027s
user    0m0.960s
sys     0m0.060s
-------------------------

[(1) & (2) were executed as normal user which can not generate files in the
4Suite directories. The first call of python in the examples was used to
generate the byte-code; then the plain execution-time can be measured at
the second one.]

You can see that (2) needs the most time; probably python generates the
optimized byte-code before it tests if it can be written. Because user can
not write into 4Suite-directories, this is a wasted effort.

When root executes (3) the .pyo files can be generated an a subsequent
repeating of (2) (#4) gives better times.



I suggest to generate optimized .pyo files and include them into the
package. This can be reached e.g. by calling

| python -O /usr/lib/python1.5/compileall.py \
|           $RPM_BUILD_ROOT/usr/lib/python1.5/site-packages
in the %install stage.

Comment 1 Trond Eivind Glomsrxd 2001-08-15 17:08:33 UTC
Later, perhaps. Not now.

Comment 2 Enrico Scholz 2001-08-15 17:26:19 UTC
What shall I do between now and "later"? 

Have I to omit the '-O' option manually or will it patched out in the
python-package?

How shall I handle rpm-packages providing .pyo files? When using them, the same
slowdown would happen. Or, will the .pyo-support completely removed from python?


BTW: it's not an enhancement; it's a bug impacting performance significantly.



Comment 3 Trond Eivind Glomsrxd 2001-08-15 18:19:28 UTC
They are built with their standard, included build procedures. Adding .pyo files
would require changing the standard buildprocedure, rebuild and retesting, which
we're avoiding as much as possible at this stage. Doing it for a little
performance gain isn't happening now - for the next release, it sounds like a
good suggestion. Not for this one.

BTW: The best way to solve this is to make the standard buildprocedure (from
distutils) create this file. I suggest you contact them on making it a feature
there.



Comment 4 Enrico Scholz 2001-09-20 18:42:43 UTC
Updating to python-2.2 has shown a new issue:

When root runs a python-program the compiled/optimized files will be created
automatically. Therefore, when updating python-1.5 to python-2.2 there will be
kept a lot of ghost files in the python-1.5/site-lisp directory.

E.g. when 4Suite was used by root, the file 
| /usr/lib/python1.5/site-packages/xml/xpath/XPath.pyo 
can be created. When updating to a python2.2 version of 4Suite, these files will
move into 
| /usr/lib/python2.2/site-packages

XPath.pyo will not be deleted because it is not in the 4Suite-package manifest.
Therefore, the old python1.5 directory will contain a lot of orphaned .pyo files
and their directories.

Comment 5 Trond Eivind Glomsrxd 2001-12-05 22:46:10 UTC
Added in 4Suite 0.11-4


Note You need to log in before you can comment on or make changes to this bug.