Bug 160125 - Unreasoable memory usage by rpm
Unreasoable memory usage by rpm
Status: CLOSED DUPLICATE of bug 155730
Product: Fedora
Classification: Fedora
Component: rpm (Show other bugs)
3
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Paul Nasrat
Mike McLean
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-06-10 19:11 EDT by H.J. Lu
Modified: 2007-11-30 17:11 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-08-16 11:03:33 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
A testcase (8.69 KB, text/plain)
2005-06-10 19:14 EDT, H.J. Lu
no flags Details
A testcase (2.86 KB, text/plain)
2005-06-10 20:39 EDT, H.J. Lu
no flags Details

  None (edit)
Description H.J. Lu 2005-06-10 19:11:00 EDT
After installing RC 3 on my EM64T machine with 2GB RAM, I tried
to install updates. The rpm used more than 1.6GB memory when doing
that.
Comment 1 H.J. Lu 2005-06-10 19:14:12 EDT
Created attachment 115317 [details]
A testcase

Here is the list of rpms I was trying to update. I did

# rpm -Uvh `cat /tmp/update.list`

The memory usage of rpm went from 10MB to 33MB, then jump to 600MB, then 1.8G:

  PID USER	PR  NI	VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
13524 root	18   0 1168g 1.7g 1.7g R 51.9 89.3   0:15.31 rpm

I had to kill it.
Comment 2 Jeff Johnson 2005-06-10 19:56:04 EDT
This is possibly an attempt to calculate md5 on a sparse file marked with %config.
The fix for that cannot be handled through normal batch upgrades, as the already installed
rpm has the problem. The fix is also isolated to the /var/log/lastlog by strcmp on path

Hmmm, I don't see the setup package within the list of packages. Perhaps there's another sparse file 
marked with %config within the set. If so, that will need treatment like /var/log/lastlog.
Comment 3 H.J. Lu 2005-06-10 20:39:14 EDT
Created attachment 115321 [details]
A testcase

Here is a shorter list. I will keep searching.
Comment 4 Jeff Johnson 2005-06-10 21:24:12 EDT
Thanks. If the mechanism I think, look for a sparse file marked %config. The code path includes all files
on the file system from old packages as well as the install set that triggers.

strace shows an attempt to mmap() the sparse file, grep'ping out all the mmap calls
might confirm if attempting a digest on a sparse file is actually the cause. Perhaps
easier than binary search on the installed files.
Comment 5 H.J. Lu 2005-06-11 12:35:45 EDT
Yes, it is /var/log/lastlog:

-r--------  1 root root 1254130450140 Jun 11 09:28 /var/log/lastlog
Comment 6 Jeff Johnson 2005-06-11 14:09:05 EDT
Upgrading rpm to version that avoids md5 digest on sparse /var/log/lastlog
marked with %config is first step.

Upgrading to a setup package that removes the %config marker is the 2nd step.

The complexities come from attempting to do both these steps at once, as
the old version of rpmlib (wich does not avoid /var/log/lastlog digest) and the old
version of setup (which has the %config marker on /var/log/lastlog) are both
involved in a singles step upgrade.

Divide and conquer ...
Comment 7 H.J. Lu 2005-06-11 14:12:36 EDT
I couldn't find the new updated rpm for FC3. The only one I found for FC3 is
rpm-4.3.2-21.
Comment 8 Jeff Johnson 2005-06-11 14:31:54 EDT
No idea where the patch that fixes /var/log/lastlog has wandered. Verify with
    strings /usr/lib/librpm.a | grep lastlog
If the library contains the string "/var/log/lastlog", then you have the fix.
Comment 9 H.J. Lu 2005-06-11 14:38:53 EDT
There is no /usr/lib/librpm.a in rpm-4.3.2-21. BTW, rpm-4.3.2-21 is the only
rpm for FC3. There is no update since the initial FC3 release.
Comment 10 Jeff Johnson 2005-06-11 14:57:20 EDT
Then look at /usr/lib/librpm*.so

And if on an elf64 system, /usr/lib64 instead.

Comment 11 H.J. Lu 2005-06-11 15:02:11 EDT
I got

[hjl@gnu-9 kernel-FC3]$  strings /usr/lib64/librpm.so | grep lastlog
[hjl@gnu-9 kernel-FC3]$
Comment 12 Jeff Johnson 2005-06-11 15:06:43 EDT
Then that version of rpm does not have the fix.

Try rpm-4.4.1-21 packages from FC4, that almost certainly has the fix (can't check, sorry).
Comment 13 Jeff Johnson 2005-06-11 15:10:46 EDT
Renaming /var/log/lastlog while installing a version of setup that does
not include the %config marker.

The command "rpm -qc setup" or "rpm -qcp setup*.rpm" will spew
which files have %config markers.
Comment 14 Paul Nasrat 2005-08-16 11:03:33 EDT
An update has been released for this issue - please update rpm* in FC3.

*** This bug has been marked as a duplicate of 155730 ***

Note You need to log in before you can comment on or make changes to this bug.