Bug 284011 - (FFlangpacks) firefox takes unacceptably long time to start
firefox takes unacceptably long time to start
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: firefox (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Martin Stransky
Fedora Extras Quality Assurance
: FutureFeature, Reopened, Triaged
: 515153 556020 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-09-09 10:31 EDT by Graham Cole
Modified: 2012-08-06 10:05 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
: 480603 (view as bug list)
Environment:
Last Closed: 2012-08-06 10:05:09 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
strace of firefox startup when issue occurs (753.07 KB, application/x-gzip)
2007-09-09 10:31 EDT, Graham Cole
no flags Details
first version (4.96 KB, patch)
2009-11-19 10:49 EST, Martin Stransky
no flags Details | Diff
second one (7.15 KB, patch)
2009-11-23 10:22 EST, Martin Stransky
caillon: review-
Details | Diff

  None (edit)
Description Graham Cole 2007-09-09 10:31:12 EDT
Description of problem:
Firefox often takes an unacceptable length of time (>60 seconds) to start for
users in our lab of Fedora 7 machines.

How reproducible:
Always reproducible. Although the steps may seem contrived, they create a
condition which is frequently encountered in day-to-day use of our lab (see
notes below)

Steps to Reproduce:
1. # touch /usr/lib/firefox-2.0.0.5/extensions/langpack*
2. $ firefox
  
Actual results:
Nothing happens... then 70 seconds later a Firefox window appears :-/

Expected results:
Firefox to appear within about 10 seconds of asking


Additional info:
Users' homedirs are mounted over NFS in our setup. 

Rebuilding the extensions cache seems to generate an unnecessarily large number
of writes to files in the profile dir, which I believe is the reason this
operation takes so long.

Rebuilding the cache on startup happens routinely in our setup whenever a user
logs in to a machine other than the last one they used. All the machines are
identically configured, so this really shouldn't be necessary. 

From studying the source of the extension manager and the resulting cache files
I notice that the metric which causes firefox to rebuild the extensions cache is
the mtime of the directory containing the extension. Whilst the files inside the
directories have identical mtimes (having been installed from the same RPM), the
same directory on different machines always has differing mtimes, depending only
on when the RPM was installed.

Cutting down the number of extensions installed (removing all the un-needed
language packs) helps immensely. Fedora's policy of bundling them all in the
same RPM helps us less so.

This problem also affects Thunderbird in exactly the same way.
Comment 1 Graham Cole 2007-09-09 10:31:12 EDT
Created attachment 191071 [details]
strace of firefox startup when issue occurs
Comment 2 Graham Cole 2007-10-03 13:13:46 EDT
Whilst I don't really have a clue how to trace and profile the Javascript bits
of firefox, I got quite far sticking a few extra debug messages in to
/usr/lib/firefox-2.0.0.5/components/nsExtensionManager.js.

My finding is that _updateManifests() is called 87 times during startup when all
the extensions' mtimes have "changed", with an average time per call of 477ms.
The vast majority of the calls come from _setOp(), so that was 40 seconds spent
mostly re-writing the same data with some temporary state information added.

This could be done a lot more efficiently, and is probably worth taking upstream.

As for preventing this process of rebuilding extensions from occurring so often
in environments such as ours, a few schemes seem possible: altering the logic to
check the mtime of files within an extension directory rather than the directory
itself, which will match up on every machine; setting the mtime of extension
directories in an RPM post-install script; and setting the
extensions.ignoreMTimeChanges preference to true by default.

I believe setting the ignoreMTimeChanges preference has the consequence of
preventing firefox from noticing extensions upgraded by external processes even
in user profile dirs, so I think it is best avoided.
Comment 3 Matěj Cepl 2008-02-21 17:35:09 EST
At this point, we're going to only be taking security fixes and major stability
fixes into this release of Fedora.  However, we still want to ensure the bug is
fixed in the next version.  We'd appreciate if you could test Firefox 3,
available at http://www.mozilla.com/en-US/firefox/all-beta.html or now shipping
as the default in Fedora rawhide and provide feedback as to whether it still
exists so we can file a ticket upstream to try to fix it in Firefox 3 before it
is released.
Comment 4 Matěj Cepl 2008-02-21 17:36:32 EST
At this point, we're going to only be taking security fixes and major stability
fixes into this release of Fedora.  However, we still want to ensure the bug is
fixed in the next version.  We'd appreciate if you could test Firefox 3,
available at http://www.mozilla.com/en-US/firefox/all-beta.html or now shipping
as the default in Fedora rawhide and provide feedback as to whether it still
exists so we can file a ticket upstream to try to fix it in Firefox 3 before it
is released.
Comment 5 Graham Cole 2008-02-26 21:13:44 EST
(In reply to comment #3)
> We'd appreciate if you could test Firefox 3,

I had a look at testing this under Firefox 3. It's presently not simple to
reproduce, as neither the upstream distribution nor the Fedora rawhide package
include the cornucopia of langpack extensions which bring out the delays.

I believe this is still an issue with Firefox 3 though, as when I tried copying
the langpack extensions from fc8 Firefox (tweaking the install.rdf file to make
them "compatible" with Firefox 3) the delays measured in the case noted in
earlier comments were roughly the same as before. So still a problem, unless the
langpacks aren't expected to be included in the fc9 Firefox package(?)

Comment 7 Christopher Aillon 2008-02-28 17:12:05 EST
Graham, I'll be adding langpacks back in to FF3 soon in rawhide, however they
should have %lang attributes in the %files list so you should be able to exclude
them by tweaking %_install_langs (I think that's the right one) in your rpmmacros
Comment 8 Richard Cunningham 2008-03-14 13:17:29 EDT
We have had this problem for sometime with RHEL4 and RHEL5, based on Graham's
comments we found that the
~/.mozilla/firefox/{weird_string}.default/extensions.cache 
file stores the modification times of the
directories in 
/usr/lib/firefox-1.5.0.12/extensions

I wrote a script which fixes the dates to the build date of the rpm. I have run
it on our machines. The startup time when moving to a new machine has been
reduced from over 1 minute before the change to about 5 or 6 seconds after the
change.

Here is the script:

for package in firefox thunderbird
do
        ver=`rpm -q $package`
        ver2=`rpm -q $package --qf '%{NAME}-%{VERSION}'`
        unixtime=`rpm -q --qf "%{BUILDTIME}\n" $package`
        date=`date -d "1970-01-01 UTC $unixtime seconds"`
        for dir in /usr/lib*/$ver2/extensions/*
        do
                touch -d "$date" "$dir"
        done
done
Comment 9 Steve Hill 2008-05-13 11:11:53 EDT
Couldn't the langpacks be packaged in separate RPMs?  I can't imagine very many
people feel the need to install every language in existence on their workstation...
Comment 10 Richard Cunningham 2008-05-13 12:23:42 EDT
Though if you work in a university as I do where people attend from many
different countries this is necessary. We have been using the fix I posted for
sometime and this fixes the issue and I think it could be rolled into the RPM
quite easily so I  don't see the need for an alternative fix to this.
Comment 11 Christopher Aillon 2008-05-13 12:58:09 EDT
See comment #7 from me.  Again, you can tell RPM to only install the languages
you want.  If you don't want to install every language, then don't.  All other
packages in the distro (with one exception) ship every language.  Simply tweak
%_install_langs and you don't even have to worry about running scripts such as
the one in comment #8.
Comment 12 Richard Cunningham 2008-05-13 13:22:15 EDT
If the RPM was modified to actually set the modified dates on the files in
/usr/lib/firefox-1.5.0.12/extensions
then none of this would be necessary, other RPMs seem to do this. I don't see
why anyone should have to do the steps in #7 or #8 (mine) to get this to work
properly so not having this done automatically for people makes it a bug.

We have found that solving this issue has taken a significant amount of time as
it is not clear at first weather it is a file server tuning issue, network or as
in this case a application issue (which we though was by far the least likely).
This would have been avoided if this bug was simply fixed and I don't understand
why there is so much resistance to actually doing so.

There are also similar issues to this in KDE such as this one:
http://bugs.kde.org/show_bug.cgi?id=104681
Comment 13 Bug Zapper 2008-05-13 23:12:59 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 14 Chris Jones 2009-01-19 07:33:01 EST
This bug is still present in F10, perhaps should be filed against devel?
Comment 15 Zdenek Kabelac 2009-01-28 07:20:33 EST
Maybe this script could help:
It should cleanup up lenghty database files

--
#!/bin/sh

cd ~/.mozilla/firefox/*.default/
for db in *.sqlite ; do
    bzip2 --best --stdout $db >$db.bz2
    sqlite3 $db 'VACUUM;'    
done
cd -
--
Comment 16 Matěj Cepl 2009-01-28 08:53:25 EST
(In reply to comment #14)
> This bug is still present in F10, perhaps should be filed against devel?

Well, could you reproduce it with devel?
Comment 17 Matěj Cepl 2009-03-07 15:29:24 EST
Reporter, could you please reply to the previous question? If you won't reply in one month, I will have to close this bug as INSUFFICIENT_DATA. Thank you.
Comment 18 Sitsofe Wheeler 2009-03-08 13:35:17 EDT
Matej:
Why so? I've yet to see anything that makes me think it won't be in F11 and when it gets tested with that we'll be back in the same case and someone is going to have to twiddle the version number (again). Believe me - there are people watching this. If it goes away in a full release we will most likely close this bug ourselves so I ask: could you be more lenient and only prompt on this after every other release of Fedora or can YOU test if it's reproducible in devel and then mark it as such (the bug seems to be well described so the result should be clear)?
Comment 19 Matěj Cepl 2009-03-09 09:29:22 EDT
(In reply to comment #18)
> Why so? I've yet to see anything that makes me think it won't be in F11 and

I am sorry, this is my mistake -- too many bugs in NEEDINFO, that I haven't really dig into this bug and thought that my request in comment 16 actually makes sense.

Assigning to developer.
Comment 20 Bug Zapper 2009-11-18 03:12:24 EST
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 21 Martin Stransky 2009-11-19 10:49:37 EST
Created attachment 370395 [details]
first version

With this patch all language packs are moved to /usr/lib/firefox-version/langpacks and only a selected one is linked to extension dir in ~/.mozilla/extensions/{ffid}. 

This hack exposes only one language pack to firefox so the start-up time is shortened.
Comment 22 Christopher Aillon 2009-11-19 12:28:52 EST
Comment on attachment 370395 [details]
first version

In general, I'm okay with this approach.  I had tossed this idea around back in ~2005, though that was before we had the system langpack support, so the main reason I ultimately decided against it was that it would need a setuid binary to do it.  Obviously, we no longer need that, so this approach will work.

A few comments though...

>+# If firefox is not running then set-up appropriate language pack
>+$MOZ_XUL_DIR/mozilla-xremote-client -a firefox 'ping()' > /dev/null 2>&1
>+if [ $? -ne 0 ]; then
>+  # clear existing locale settings
>+  mkdir -p $MOZ_EXTENSIONS_PROFILE_DIR
>+  rm -rf $MOZ_EXTENSIONS_PROFILE_DIR/langpack-*@firefox.mozilla.org

How would this affect langpacks that are installed by the user?


> # set up the firefox start script
>+XUL_DIR=`pkg-config --variable=libdir libxul`
> %{__rm} -rf $RPM_BUILD_ROOT%{_bindir}/firefox
>-%{__cat} %{SOURCE21} | %{__sed} -e 's,FIREFOX_VERSION,%{internal_version},g' > \
>+%{__cat} %{SOURCE21} | %{__sed} -e 's,FIREFOX_VERSION,%{internal_version},g' \
>+		     | %{__sed} -e "s,XULRUNNER_DIRECTORY,$XUL_DIR,g" > \

Variable confusion here :)
Comment 23 Martin Stransky 2009-11-23 10:22:52 EST
Created attachment 373143 [details]
second one

It enables local (user installed) langpacks at the profile extension directory when MOZ_LOCAL_LANGPACKS is set.
Comment 24 Martin Stransky 2009-11-25 10:19:18 EST
Added to firefox-3.6.1-0.4.b3.fc13
Comment 25 Christopher Aillon 2009-12-07 14:54:08 EST
Comment on attachment 373143 [details]
second one

>+##
>+## Fedora enables you to install custom language packs at firefox extension
>+## directory (specified by MOZ_EXTENSIONS_PROFILE_DIR). To enable Firefox 
>+## custom localization, set MOZ_LOCAL_LANGPACKS=1 in your environment
>+## before launching Firefox.
>+##
>+#
>+# MOZ_LOCAL_LANGPACKS=1
>+# export MOZ_LOCAL_LANGPACKS

Wait, what?  This is horrible.  Users should not have to change their environment to make things work on Fedora that work on other platforms by default.  This is just a recipe for failure.
Comment 26 Christopher Aillon 2009-12-08 08:55:16 EST
Perhaps this should touch a file such as .fedora-langpack-install which contains the langpack we linked to, so we can remove that link and only that link when needed.

Re-opening to fix properly.
Comment 27 Martin Stransky 2010-01-05 08:37:44 EST
MOZ_LOCAL_LANGPACKS has been removed in firefox-3.6.1-0.8.b5.fc13
Comment 28 Richard Cunningham 2010-01-05 08:55:44 EST
I can't believe this bug is still going. There is a simple fix for this problem.

The Firefox RPM should preserve modification dates when it is installed.

This is mentioned in Graham's original description from over 2 years ago. We have been using the work around I post in comment #8 for almost 2 years now.
Comment 29 Martin Stransky 2010-03-09 09:24:09 EST
*** Bug 515153 has been marked as a duplicate of this bug. ***
Comment 30 Bug Zapper 2010-03-15 07:55:06 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 13 development cycle.
Changing version to '13'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 31 William Lovaton 2010-07-19 22:37:38 EDT
Is there any final resolution for this bug? I just created a symbolic link in ~/.mozilla/extensions/{firefox-id} to my preferred langpack and that worked fine but that is so not cool for an end user.

Maybe creating a firefox-lang-?? rpm package or something.
Comment 32 Ladar Levison 2010-10-29 08:46:25 EDT
The problem is that RH is shipping an unprofiled build. See this article for details on how to enable PGO:

https://developer.mozilla.org/en/building_with_profile-guided_optimization
Comment 33 Bug Zapper 2011-06-02 14:39:30 EDT
This message is a reminder that Fedora 13 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 13.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '13'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 13's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 13 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 34 Matěj Cepl 2011-06-09 09:16:18 EDT
*** Bug 556020 has been marked as a duplicate of this bug. ***
Comment 35 Martin Stransky 2012-08-06 10:05:09 EDT
I believe it's already fixed. The PGO build makes no difference here (at least when we test it last time) but we can reconsider it in future.

Note You need to log in before you can comment on or make changes to this bug.