Bug 226893 - Tcl needs a restricted auto_path
Tcl needs a restricted auto_path
Product: Fedora
Classification: Fedora
Component: tcl (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Marcela Mašláňová
Depends On: 227200 227951
  Show dependency treegraph
Reported: 2007-02-01 16:34 EST by Wart
Modified: 2008-01-18 03:18 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-01-18 03:18:36 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Patch to restrict entries in auto_path (620 bytes, patch)
2007-02-07 15:11 EST, Wart
no flags Details | Diff

  None (edit)
Description Wart 2007-02-01 16:34:20 EST
Description of problem:

Tcl searches for packages in 3 directories:
 * /usr/share/tcl8.5
 * /usr/share
 * /usr/lib

The search algorithm involves going into each subdirectory of these three
directories, looking for a file named 'pkgIndex.tcl', and then reading it. 
Since there are quite a few subdirectories in /usr/share and /usr/lib that
aren't tcl-related, this results in a lot of wasted time and effort.

To fix, the default value for 'auto_path' in Tcl needs to be reduced to 2
directories:  /usr/lib{64}/tcl8.x and /usr/share/tcl8.x.  My informal tests have
shown that this can reduce the initial load time for Tcl scripts from 3.7s to 0.2s.

Tcl extension packages in Fedora would also have to be modified to install into
these version-specific tcl directories, instead of the current convention of
/usr/lib, /usr/share.  A 'tcl-sitearch' macro would also be needed in Tcl
package spec files to detect the correct install directory.  This would bring
Tcl more in line with the directory structure used by other scripting languages
(perl, python) in Fedora, and give Tcl applications a boost in startup times.

The extensions can be modified to use the new install directories before the
core Tcl package has changed.  This won't help the startup time, but will
prevent 'package not found' errors if Tcl changes before the extensions.

Version-Release number of selected component (if applicable):
tcl-8.5a5-5.fc7 and all earlier versions

How reproducible:

Steps to Reproduce:
1. Put the following in a file named 'tclpath.tcl' (note that line 2 is
commented out):
#set auto_path [list /usr/share/tcl8.5 /usr/lib/tcl8.5]
package require http

2. Time how long it takes to execute the script:  'time tclsh tclpath.tcl'
3. Reboot to flush the kernel and disk memory caches
3. Uncomment line 2 in tclpath.tcl (set auto_path ...)
4. Time how long it takes to execute the script again.
Actual results:
This takes ~3.7s on my test system for the first run, and ~.2s for the second.

Expected results:
The first run takes just as long as the second.

Additional info:

The timing may vary depending on the state of the kernel's in-memory filesystem
cache and what is in the disk's own buffer.  For more comparable tests, run the
test script as the first command after a reboot.

I haven't poked into the Tcl internals to find the best way to fix the default
package search path, but I'll post a patch if I find a decent way to do it.
Comment 1 Marcela Mašláňová 2007-02-05 09:09:32 EST
Good point, but I think the upstream should be the most capable to solve it.
Comment 2 Wart 2007-02-06 13:15:24 EST
I have contacted upstream about this, and their response was expected:  Tcl has
10 years of history of using /usr/lib and /usr/share as package directories, and
if it stops doing so then applications break.

I agree that this is fine for upstream's policy, because they have to deal with
existing installations of extensions, and with extensions installed manually. 
But Fedora is in a position to fix that for our own distribution.  Fedora can
control where Fedora packages are installed, which means we can put all of
Fedora's tcl extensions into %{tcl_sitearch} and %{tcl_sitelib} and achieve this
optimization.  If there are any Fedora applications that are hardcoding
/usr/lib, then we have the ability to fix them.

btw, I have a patch that fixes this, but I want to test it a little more before
attaching it.
Comment 3 Marcela Mašláňová 2007-02-07 05:43:27 EST
Ok, we can fix it.
Comment 4 Wart 2007-02-07 15:11:03 EST
Created attachment 147591 [details]
Patch to restrict entries in auto_path

This patch modifies init.tcl to prevent it from adding /usr/lib{64} and
/usr/share onto the package search path.  This patch should not be committed
until bug #227200 has been committed and package maintainers have had a chance
to update their packages to install into the new package directories.  It might
be best to wait for the Fedora Packaging committee to approve the proposed Tcl
packaging guidelines:  http://fedoraproject.org/wiki/MichaelThomas/Tcl
Comment 5 Marcela Mašláňová 2008-01-18 03:18:36 EST
fixed in tcl-1:8.5.0-6

Note You need to log in before you can comment on or make changes to this bug.