Bug 567252 - Memory consuming file /usr/lib/local/local-archive
Summary: Memory consuming file /usr/lib/local/local-archive
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: rawhide
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Anaconda Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-02-22 13:13 UTC by Zdenek Kabelac
Modified: 2010-04-20 14:01 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-02-25 15:41:24 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Zdenek Kabelac 2010-02-22 13:13:49 UTC
Description of problem:

Anaconda is using large file: 
/usr/lib/local/local-archive

Which is nearly 100MB large - it comes from glib-common package. However anaconda does not even have all the binaries and libs in the system - thus such file should  be generated to fit anaconda needs - i.e. files in /usr/share/local/ seems to be quite pruned having much smaller size - seems to  be installer selects only few of them to installation image - are they all?

Current solution does have several problems: file takes quite some space on the installation image. File is completely loaded into a memory when some process call 'mlockall()' and there are several such processes - lvm is amongst them. 

This basically leads to a situation the startup of any lvm command which modifies dm tables means mapping whole 100MB file into a memory.

There are several possible solutions for the problem:

- Detect all tools which are using mlockall() and use LANG=C for them - not really a good plan - but a pretty quick workaround.

- Try to make much smaller locale-archive file only for locale files which are really needed for anaconda installer - still might lead to maybe 20MB file??
(never tried - just a wild guess).

- Generate locale-archive according to locales given at the startup of installation - the best way - locale-archive should be in the range of hundreds KB, but probably requires some mount tricks to have the file generated probably in ramdisk shortly after installer asks for language.

Here is example of command, how the locale-archive file could be created:

localedef -i cs_CZ -c -f UTF-8 -A /usr/share/locale/locale.alias cs_CZ.UTF-8

Version-Release number of selected component (if applicable):
rawhide

How reproducible:
ls -la /usr/lib/local/local-archive

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Chris Lumens 2010-02-25 15:32:56 UTC
Isn't this same problem going to exist post-installation on any system with memory pressure?  Looking at my installed system, I see that /usr/lib/locale/locale-archive is nearly 100 MB there too.  If the lvm tools behave the same way in a regular environment as they do in the anaconda environment, that means they're going to lock all that memory there too.  Therefore running lvm commands on a low-memory system is going to have the same problem as running lvm commands during anaconda.

Comment 3 Zdenek Kabelac 2010-02-25 15:36:46 UTC
Yes - it works this way in localized environment for quite some time. mlockall() and 100MB mmap files are not going to work well together...

Patch for lvm is under development to avoid locking this large file in memory.

Comment 4 Chris Lumens 2010-02-25 15:41:24 UTC
Okay, cool.  Then I really don't think we need to do anything in anaconda since a patch for lvm will take care of this across the board.  Glad to hear it's in the works.

Comment 5 Zdenek Kabelac 2010-02-25 15:57:15 UTC
I assume you still could save considerable amount of disk space on anaconda image if locales would be compiled in place.....

bzip2 compress whole file into 18MB, 7zip  3MB 

But I don't know how much disk space it takes on squashfs - and also
how much memory it takes to read such file - maybe it could be more memory efficient to have this compiled in ram disk.

Comment 6 Ales Kozumplik 2010-04-20 14:01:11 UTC
Zdenek updated me today that lvm after version 2.02.62 and dmeventd, clvmd after version 2.02.63 have a workaround for the problem.


Note You need to log in before you can comment on or make changes to this bug.