Bug 226893

Summary: Tcl needs a restricted auto_path
Product: [Fedora] Fedora Reporter: Wart <wart>
Component: tclAssignee: Marcela Mašláňová <mmaslano>
Status: CLOSED RAWHIDE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: sander
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-01-18 08:18:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 227200, 227951    
Bug Blocks:    
Attachments:
Description Flags
Patch to restrict entries in auto_path none

Description Wart 2007-02-01 21:34:20 UTC
Description of problem:

Tcl searches for packages in 3 directories:
 * /usr/share/tcl8.5
 * /usr/share
 * /usr/lib

The search algorithm involves going into each subdirectory of these three
directories, looking for a file named 'pkgIndex.tcl', and then reading it. 
Since there are quite a few subdirectories in /usr/share and /usr/lib that
aren't tcl-related, this results in a lot of wasted time and effort.

To fix, the default value for 'auto_path' in Tcl needs to be reduced to 2
directories:  /usr/lib{64}/tcl8.x and /usr/share/tcl8.x.  My informal tests have
shown that this can reduce the initial load time for Tcl scripts from 3.7s to 0.2s.

Tcl extension packages in Fedora would also have to be modified to install into
these version-specific tcl directories, instead of the current convention of
/usr/lib, /usr/share.  A 'tcl-sitearch' macro would also be needed in Tcl
package spec files to detect the correct install directory.  This would bring
Tcl more in line with the directory structure used by other scripting languages
(perl, python) in Fedora, and give Tcl applications a boost in startup times.

The extensions can be modified to use the new install directories before the
core Tcl package has changed.  This won't help the startup time, but will
prevent 'package not found' errors if Tcl changes before the extensions.


Version-Release number of selected component (if applicable):
tcl-8.5a5-5.fc7 and all earlier versions

How reproducible:
Always

Steps to Reproduce:
1. Put the following in a file named 'tclpath.tcl' (note that line 2 is
commented out):
#!/usr/bin/tclsh
#set auto_path [list /usr/share/tcl8.5 /usr/lib/tcl8.5]
package require http

2. Time how long it takes to execute the script:  'time tclsh tclpath.tcl'
3. Reboot to flush the kernel and disk memory caches
3. Uncomment line 2 in tclpath.tcl (set auto_path ...)
4. Time how long it takes to execute the script again.
  
Actual results:
This takes ~3.7s on my test system for the first run, and ~.2s for the second.

Expected results:
The first run takes just as long as the second.

Additional info:

The timing may vary depending on the state of the kernel's in-memory filesystem
cache and what is in the disk's own buffer.  For more comparable tests, run the
test script as the first command after a reboot.

I haven't poked into the Tcl internals to find the best way to fix the default
package search path, but I'll post a patch if I find a decent way to do it.

Comment 1 Marcela Mašláňová 2007-02-05 14:09:32 UTC
Good point, but I think the upstream should be the most capable to solve it.

Comment 2 Wart 2007-02-06 18:15:24 UTC
I have contacted upstream about this, and their response was expected:  Tcl has
10 years of history of using /usr/lib and /usr/share as package directories, and
if it stops doing so then applications break.

I agree that this is fine for upstream's policy, because they have to deal with
existing installations of extensions, and with extensions installed manually. 
But Fedora is in a position to fix that for our own distribution.  Fedora can
control where Fedora packages are installed, which means we can put all of
Fedora's tcl extensions into %{tcl_sitearch} and %{tcl_sitelib} and achieve this
optimization.  If there are any Fedora applications that are hardcoding
/usr/lib, then we have the ability to fix them.

btw, I have a patch that fixes this, but I want to test it a little more before
attaching it.

Comment 3 Marcela Mašláňová 2007-02-07 10:43:27 UTC
Ok, we can fix it.

Comment 4 Wart 2007-02-07 20:11:03 UTC
Created attachment 147591 [details]
Patch to restrict entries in auto_path

This patch modifies init.tcl to prevent it from adding /usr/lib{64} and
/usr/share onto the package search path.  This patch should not be committed
until bug #227200 has been committed and package maintainers have had a chance
to update their packages to install into the new package directories.  It might
be best to wait for the Fedora Packaging committee to approve the proposed Tcl
packaging guidelines:  http://fedoraproject.org/wiki/MichaelThomas/Tcl

Comment 5 Marcela Mašláňová 2008-01-18 08:18:36 UTC
fixed in tcl-1:8.5.0-6