Red Hat Bugzilla – Bug 51792
Optimized byte-code should be provided also
Last modified: 2008-05-01 11:38:00 EDT
4Suite-0.11-2 contains only normal .pyc byte code but node the optimized .pyo one: | $ rpm -ql 4Suite | grep pyo | $ This leads to significant slowdowns when used in an optimized project. E.g.: ----------- $ cat test.py import xml.xslt.Processor import xml.dom; dt = xml.dom.DocumentType.DocumentType('foo', None, None, None, None) dc = xml.dom.Document.Document(dt) dc.createElement('bar') proc = xml.xslt.Processor.Processor() $ python test.py && time python test.py ## (1) real 0m4.891s user 0m0.970s sys 0m0.110s $ python -O test.py && time python -O test.py ## (2) real 0m4.002s user 0m3.930s sys 0m0.070s $ su - # python -O test.py && time python -O test.py ## (3) real 0m1.027s user 0m0.910s sys 0m0.110s $ exit $ python -O test.py && time python -O test.py ## (4) real 0m1.027s user 0m0.960s sys 0m0.060s ------------------------- [(1) & (2) were executed as normal user which can not generate files in the 4Suite directories. The first call of python in the examples was used to generate the byte-code; then the plain execution-time can be measured at the second one.] You can see that (2) needs the most time; probably python generates the optimized byte-code before it tests if it can be written. Because user can not write into 4Suite-directories, this is a wasted effort. When root executes (3) the .pyo files can be generated an a subsequent repeating of (2) (#4) gives better times. I suggest to generate optimized .pyo files and include them into the package. This can be reached e.g. by calling | python -O /usr/lib/python1.5/compileall.py \ | $RPM_BUILD_ROOT/usr/lib/python1.5/site-packages in the %install stage.
Later, perhaps. Not now.
What shall I do between now and "later"? Have I to omit the '-O' option manually or will it patched out in the python-package? How shall I handle rpm-packages providing .pyo files? When using them, the same slowdown would happen. Or, will the .pyo-support completely removed from python? BTW: it's not an enhancement; it's a bug impacting performance significantly.
They are built with their standard, included build procedures. Adding .pyo files would require changing the standard buildprocedure, rebuild and retesting, which we're avoiding as much as possible at this stage. Doing it for a little performance gain isn't happening now - for the next release, it sounds like a good suggestion. Not for this one. BTW: The best way to solve this is to make the standard buildprocedure (from distutils) create this file. I suggest you contact them on making it a feature there.
Updating to python-2.2 has shown a new issue: When root runs a python-program the compiled/optimized files will be created automatically. Therefore, when updating python-1.5 to python-2.2 there will be kept a lot of ghost files in the python-1.5/site-lisp directory. E.g. when 4Suite was used by root, the file | /usr/lib/python1.5/site-packages/xml/xpath/XPath.pyo can be created. When updating to a python2.2 version of 4Suite, these files will move into | /usr/lib/python2.2/site-packages XPath.pyo will not be deleted because it is not in the 4Suite-package manifest. Therefore, the old python1.5 directory will contain a lot of orphaned .pyo files and their directories.
Added in 4Suite 0.11-4