Bug 437914 - RFE: Yum 32bit for installer? -- as 64bit uses twice as much memory
RFE: Yum 32bit for installer? -- as 64bit uses twice as much memory
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: yum (Show other bugs)
9
x86_64 Linux
low Severity low
: ---
: ---
Assigned To: Seth Vidal
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-03-18 00:28 EDT by Paul Walmsley
Modified: 2008-10-18 06:44 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-10-16 14:07:47 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Paul Walmsley 2008-03-18 00:28:24 EDT
Description of problem:
Checking process information on anaconda during a Fedora 9 install of 1992
packages shows 822684kB peak address space usage and (over a handful of samples)
about 490000kB RSS.  A companion anaconda process appears to be using about
240MB address space also.  

Version-Release number of selected component (if applicable):
Fedora 9 alpha

How reproducible:
Install Fedora 9 alpha x86_64 and select as many packages as possible to install.

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:
Much less memory should be consumed by the installer.

Additional info:
Comment 1 Chris Lumens 2008-04-22 11:34:08 EDT
This is likely the python object size on x86_64 hurting us again.  Reassigning
to yum so they can verify that opinion.
Comment 2 James Antill 2008-04-22 12:01:11 EDT
 It almost certainly doesn't help, 800MB VSZ and 400MB RSS is still a lot
though, most of the tests/fixes I did leading upto Fed-9 showed[1] a Fed-8 =>
Fed-9 upgrade taking about half of that (depending on what repos. you had
enabled) ... is anaconda doing anything like creating two YumBase instances? Or
having a lot of repos. with data in them?

 In all the testing I've done 64bit python seems to take roughly 4x the memory
as 32bit python ... so if we install via. a 32bit python/libs. we'd probably get
200MB VSZ / 100MB RSS ... but it's probably too late for even that?

[1] We had to backout some of the memory fixes due to unicode issues, but
nothing that would give 2x usage.
Comment 3 Bug Zapper 2008-05-14 02:40:34 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 4 Jan Hutař 2008-08-06 08:12:06 EDT
(In reply to comment #2)
>  In all the testing I've done 64bit python seems to take roughly 4x the memory
> as 32bit python ... so if we install via. a 32bit python/libs. we'd probably get
> 200MB VSZ / 100MB RSS ... but it's probably too late for even that?

Could we try this for F10, or is it too late for that?
Comment 5 Seth Vidal 2008-08-06 08:27:57 EDT
much too late, I think
Comment 6 Charles R. Anderson 2008-10-16 10:56:40 EDT
Are you sure that the 4x VSZ isn't really due to the reserved mappings:

http://linux.derkeiler.com/Mailing-Lists/Kernel/2007-04/msg08907.html
Comment 7 James Antill 2008-10-16 13:53:20 EDT
After I posted comment #2 I did some more investigation, and found out some more data: http://illiterat.livejournal.com/4615.html is the basic summary.
 Reserved mappings are a big part of the problem, but there are other things like mapping all of locale-archive.

 Which basically make VSZ worthless on x86_64, so I think we should just concentrate on RSS ... which is pretty close to the "expected" 2x over .i386.
Comment 8 Jeremy Katz 2008-10-16 14:07:47 EDT
Doing this just for the installer is not going to happen.  It would have to be generally done for the distro as a whole.  We're trying to reduce the number of special case hacks in the installer, not add whole new levels of insane ones.
Comment 9 James Antill 2008-10-16 14:17:25 EDT
So to expand on that, after speaking with Jeremy, we have these problems:

1. Parts of the installer _must_ be in .x86_64, as they speak to the kernel (think modprobe) ... so we can't just use a full .i386 stage2.

2. There is no full multilib. python, so we can't just pull in the .i386 bits from the .x86_64 tree.

3. Pulling in bits of the installer from the .i386 and bits from the .x86_64 tree, is insanity.

4. Having _just_ a python.i386 in the x86_64 tree wouldn't work because then python extensions on .x86_64 programs won't work (apache/x-chat/etc.)

...so the only possible solution is to have a full multilib. python in Fedora, and then anaconda could just choose the .i386 version, but that is non-trivial and probably has no chance of happening until we have at least one full time python maintainer working with upstream (probably more).

 Ahh, well.
Comment 10 Jeremy Katz 2008-10-16 14:27:21 EDT
(In reply to comment #9)
...so the only possible solution is to have a full multilib. python in Fedora,
> and then anaconda could just choose the .i386 version, but that is non-trivial
> and probably has no chance of happening until we have at least one full time
> python maintainer working with upstream (probably more).

And even then, it's *extremely* unlikely because then anaconda would be using a different python on x86_64 than everything else that we ship which all but guarantees weird bugs that are seen nowhere other than the installer.  And really, better ways to spend time than tracking down things like that.

Note You need to log in before you can comment on or make changes to this bug.