Bug 451607

Summary: Updating system causes massive memory leak
Product: [Fedora] Fedora Reporter: Stewart Adam <s.adam>
Component: yum-prestoAssignee: Jonathan Dieter <jonathan>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: high    
Version: 9CC: bobgus, john5342, tim.lauridsen
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-02-04 08:27:21 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stewart Adam 2008-06-16 03:53:55 UTC
Description of problem:
Running 'yum update' with yum-presto enabled in certain cases can cause a
massive memory leak causing the system to become unresponsive. Running yum with
--disablepresto fixes the problem.

Version-Release number of selected component (if applicable):
yum-3.2.16-2.fc9.noarch
yum-presto-0.4.5-1.fc9.noarch

How reproducible:
Sometimes - I've never seen this happen before, but with this specific package
set I was able to reproduce it every time I ran 'yum update' with presto.

Steps to Reproduce:
1. Setup this script and start it as root:

#!/bin/sh
while true;do
  ps -A -o comm,pmem,pid | while read line;do
    if [ $(echo "$line" | awk '{printf int($2)}') -gt 80 ];then
      kill -9 `echo $line | awk '{printf $3}'`
    fi
  done
  sleep 1
done

2. Run "yum update -x libpurple -x NetworkManager\*"
  
Actual results:
Yum is killed because it uses over 80% of the system's memory (otherwise, system
RAM is used up until there is nothing left and system becomes unresponsive)

Expected results:
No memory leak

Additional info:
I have 2GB of RAM. This is the yum session that provoked the memory leak:

[user@host ~]$ sudo yum update -x libpurple -x Network\* --disablepresto
Loaded plugins: presto, refresh-packagekit
Excluding Packages in global exclude list
Finished
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package kernel-headers.x86_64 0:2.6.25.6-55.fc9 set to be updated
---> Package buildsys-build-rpmfusion.x86_64 9:9.1-6.lvn9 set to be updated
---> Package hal-libs.x86_64 0:0.5.11-2.fc9 set to be updated
---> Package xorg-x11-server-common.x86_64 0:1.4.99.902-3.20080612.fc9 set to be
updated
---> Package hal-libs.i386 0:0.5.11-2.fc9 set to be updated
---> Package bzr.x86_64 0:1.5-2.fc9 set to be updated
--> Processing Dependency: python-pycurl for package: bzr
---> Package libdvdnav.x86_64 0:4.1.2-1.fc9 set to be updated
---> Package python-sqlalchemy.noarch 0:0.4.6-1.fc9 set to be updated
---> Package hal.x86_64 0:0.5.11-2.fc9 set to be updated
---> Package postgresql-libs.x86_64 0:8.3.3-1.fc9 set to be updated
---> Package kernel.x86_64 0:2.6.25.6-55.fc9 set to be installed
---> Package xorg-x11-server-Xorg.x86_64 0:1.4.99.902-3.20080612.fc9 set to be
updated
---> Package logrotate.x86_64 0:3.7.6-5.fc9 set to be updated
---> Package java-1.6.0-openjdk-plugin.x86_64 1:1.6.0.0-0.15.b09.fc9 set to be
updated
---> Package python-virtinst.noarch 0:0.300.3-7.fc9 set to be updated
---> Package device-mapper-multipath.x86_64 0:0.4.7-15.fc9 set to be updated
---> Package postgresql-python.x86_64 0:8.3.3-1.fc9 set to be updated
---> Package hal-devel.x86_64 0:0.5.11-2.fc9 set to be updated
---> Package xorg-x11-server-Xvfb.x86_64 0:1.4.99.902-3.20080612.fc9 set to be
updated
---> Package buildsys-build-rpmfusion-kerneldevpkgs-current.x86_64 9:9.1-6.lvn9
set to be updated
---> Package libdvdread-devel.x86_64 0:4.1.2-1.fc9 set to be updated
---> Package buildsys-build-rpmfusion-kerneldevpkgs-newest.x86_64 9:9.1-6.lvn9
set to be updated
---> Package kernel-devel.x86_64 0:2.6.25.6-55.fc9 set to be installed
---> Package grip.x86_64 1:3.2.0-19.fc9 set to be updated
---> Package libdvdread.x86_64 0:4.1.2-1.fc9 set to be updated
---> Package bluez-libs.x86_64 0:3.32-1.fc9 set to be updated
---> Package pykickstart.noarch 0:1.38-2.fc9 set to be updated
---> Package java-1.6.0-openjdk.x86_64 1:1.6.0.0-0.15.b09.fc9 set to be updated
---> Package kpartx.x86_64 0:0.4.7-15.fc9 set to be updated
---> Package gdb.x86_64 0:6.8-10.fc9 set to be updated
---> Package bogofilter.x86_64 0:1.1.7-1.fc9 set to be updated
---> Package xen-libs.x86_64 0:3.2.0-12.fc9 set to be updated
--> Running transaction check
---> Package python-pycurl.x86_64 0:7.16.4-3.fc9 set to be updated
--> Finished Dependency Resolution
--> Running transaction check
---> Package kernel-devel.x86_64 0:2.6.25.4-30.fc9 set to be erased
---> Package kernel.x86_64 0:2.6.25.4-30.fc9 set to be erased
--> Processing Dependency: kernel-uname-r = 2.6.25.4-30.fc9.x86_64 for package:
kmod-fglrx-2.6.25.4-30.fc9.x86_64
--> Running transaction check
---> Package kmod-fglrx-2.6.25.4-30.fc9.x86_64.x86_64 0:8.493-1.8.05.fc9 set to
be erased
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size 
=============================================================================
Installing:
 kernel                  x86_64     2.6.25.6-55.fc9  updates            18 M
 kernel-devel            x86_64     2.6.25.6-55.fc9  updates           5.2 M
Updating:
 bluez-libs              x86_64     3.32-1.fc9       updates            58 k
 bogofilter              x86_64     1.1.7-1.fc9      updates           472 k
 buildsys-build-rpmfusion  x86_64     9:9.1-6.lvn9     livna              11 k
 buildsys-build-rpmfusion-kerneldevpkgs-current  x86_64     9:9.1-6.lvn9    
livna             8.0 k
 buildsys-build-rpmfusion-kerneldevpkgs-newest  x86_64     9:9.1-6.lvn9    
livna             7.9 k
 bzr                     x86_64     1.5-2.fc9        updates           5.5 M
 device-mapper-multipath  x86_64     0.4.7-15.fc9     updates           440 k
 gdb                     x86_64     6.8-10.fc9       updates           3.4 M
 grip                    x86_64     1:3.2.0-19.fc9   updates           436 k
 hal                     x86_64     0.5.11-2.fc9     updates           500 k
 hal-devel               x86_64     0.5.11-2.fc9     updates            32 k
 hal-libs                x86_64     0.5.11-2.fc9     updates            67 k
 hal-libs                i386       0.5.11-2.fc9     updates            65 k
 java-1.6.0-openjdk      x86_64     1:1.6.0.0-0.15.b09.fc9  updates            27 M
 java-1.6.0-openjdk-plugin  x86_64     1:1.6.0.0-0.15.b09.fc9  updates         
  29 k
 kernel-headers          x86_64     2.6.25.6-55.fc9  updates           715 k
 kpartx                  x86_64     0.4.7-15.fc9     updates            21 k
 libdvdnav               x86_64     4.1.2-1.fc9      updates            84 k
 libdvdread              x86_64     4.1.2-1.fc9      updates            52 k
 libdvdread-devel        x86_64     4.1.2-1.fc9      updates            18 k
 logrotate               x86_64     3.7.6-5.fc9      updates            52 k
 postgresql-libs         x86_64     8.3.3-1.fc9      updates           212 k
 postgresql-python       x86_64     8.3.3-1.fc9      updates            69 k
 pykickstart             noarch     1.38-2.fc9       updates           248 k
 python-sqlalchemy       noarch     0.4.6-1.fc9      updates           1.4 M
 python-virtinst         noarch     0.300.3-7.fc9    updates           193 k
 xen-libs                x86_64     3.2.0-12.fc9     updates           157 k
 xorg-x11-server-Xorg    x86_64     1.4.99.902-3.20080612.fc9  updates         
 2.4 M
 xorg-x11-server-Xvfb    x86_64     1.4.99.902-3.20080612.fc9  updates         
 1.6 M
 xorg-x11-server-common  x86_64     1.4.99.902-3.20080612.fc9  updates         
  70 k
Removing:
 kernel                  x86_64     2.6.25.4-30.fc9  installed          70 M
 kernel-devel            x86_64     2.6.25.4-30.fc9  installed          33 M
Installing for dependencies:
 python-pycurl           x86_64     7.16.4-3.fc9     fedora             77 k
Removing for dependencies:
 kmod-fglrx-2.6.25.4-30.fc9.x86_64  x86_64     8.493-1.8.05.fc9  installed     
   2.8 M

Transaction Summary
=============================================================================
Install      3 Package(s)         
Update      30 Package(s)         
Remove       3 Package(s)         

Total size: 69 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test <-- Normally fails here unless --disablepresto is used
Finished Transaction Test
Transaction Test Succeeded
Running Transaction

Comment 1 Jonathan Dieter 2008-06-18 15:32:13 UTC
I honestly don't know what to do with this.  Presto runs code at predownload
(i.e. just after you press y, but before "Downloading Packages" appears), and
again at post-transaction (i.e. just after all the packages have been
installed).  Absolutely *no* presto code is running between those two points.

Having said that, if running yum with --disablepresto fixes the problem then it
almost *has* to be a bug in presto.

I have been unable to reproduce the problem.  If you can reproduce, can you run
"yum -d 10 update" and attach the output?  Thanks.

Comment 2 Stewart Adam 2008-06-18 19:30:59 UTC
I've updated twice since then with presto (yum's even running now) without any
problems, so the problem seems to be somehow related to that specific package
set. Since I've already upgraded it would be really long to downgrade and try to
upgrade again... Is it okay if we close this as NOTABUG and I'll reopen if it
happens again (and I have debug output)?

Comment 3 Bob Gustafson 2008-06-26 21:41:52 UTC
I don't know if this is related, but I updated to a new kernel (Fed8) and this
kernel is unusable because of a memory leak. I have 3GB and it takes about 10-20
minutes to fill, then I need to reboot.

I went back a kernel using the boot/grub.conf option.

The kernel with a problem is 2.6.25.6-27.fc8
The kernel that runs ok   is 2.6.25.4-10.fc8

I can re-submit this somewhere else if you would like. Also can provide
additional information.

There is a memory leak out there..

Comment 4 john5342 2008-08-03 00:09:42 UTC
I saw this problem myself.

The problem was specifically related to one of the java packages. I forget exactly which one but i updated all other updates at the time seperately and was left with just one specific java package which caused everything to freeze as described. When i disabled presto the normal full package was downloaded and everything installed fine.

I would guess either the delta rpm in the presto repo was corrupt on the server or was generated wrongly somehow.

Have not had a problem since though

Comment 5 Jonathan Dieter 2009-01-24 18:26:30 UTC
Have there been any problems since the last comment?  If there haven't, I think I'm going to close this as NOTABUG in a day or two.

Comment 6 john5342 2009-01-24 18:46:47 UTC
As per comment #4 I had exactly the same problem and haven't seen any hints of the same problem again since. Have not tested with f10 though as there doesn't appear to be an x86_64 repo last i checked.

Comment 7 Stewart Adam 2009-01-24 18:51:37 UTC
I'm in the same boat, no DRPM repo for F10 x86_64 but I've had no problems since that one incident a while back on the i686 machines I test on.

Comment 8 Jonathan Dieter 2009-02-04 08:27:21 UTC
Ok, closing as NOTABUG.  If you have any problems, please reopen.