Bug 212504 - Yum hangs while reading /var/lib/rpm/Packages
Yum hangs while reading /var/lib/rpm/Packages
Status: CLOSED DUPLICATE of bug 213963
Product: Fedora
Classification: Fedora
Component: rpm (Show other bugs)
6
All Linux
medium Severity high
: ---
: ---
Assigned To: Paul Nasrat
:
: 214129 220685 220987 220988 221044 221053 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-10-26 22:20 EDT by Greg Martyn
Modified: 2014-01-21 17:55 EST (History)
19 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-07-17 08:40:13 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
last 30 lines of strace (1.66 KB, text/plain)
2006-10-26 22:20 EDT, Greg Martyn
no flags Details

  None (edit)
Description Greg Martyn 2006-10-26 22:20:30 EDT
Description of problem:
[root@localhost yum]# yum clean all
Loading "installonlyn" plugin
Cleaning up Everything
[root@localhost yum]# yum upgrade -d 10 -e 10
Loading "installonlyn" plugin
Running "config" handler for "installonlyn" plugin
Yum Version: 3.0
COMMAND: yum -d 10 -e 10
Installroot: /
Setting up Upgrade Process
Setting up repositories
livna                     100% |=========================| 1.1 kB    00:00
livna-debuginfo           100% |=========================|  951 B    00:00
livna-source              100% |=========================|  951 B    00:00
core                      100% |=========================| 1.1 kB    00:00
extras-source             100% |=========================|  951 B    00:00
extras-debuginfo          100% |=========================|  951 B    00:00
updates                   100% |=========================| 1.2 kB    00:00
core-source               100% |=========================|  951 B    00:00
extras                    100% |=========================| 1.1 kB    00:00
core-debuginfo            100% |=========================| 1.1 kB    00:00
Reading repository metadata in from local files
Setting up Package Sacks
primary.xml.gz            100% |=========================|  80 kB    00:00
################################################## 198/198
primary.xml.gz            100% |=========================|  19 kB    00:00
################################################## 106/106
primary.xml.gz            100% |=========================|  30 kB    00:00
################################################## 112/112
primary.xml.gz            100% |=========================| 824 kB    00:00
################################################## 2242/2242
primary.xml.gz            100% |=========================| 617 kB    00:05
################################################## 2271/2271
primary.xml.gz            100% |=========================| 250 kB    00:03
################################################## 1516/1516
primary.xml.gz            100% |=========================|  16 kB    00:00
################################################## 44/44
primary.xml.gz            100% |=========================| 305 kB    00:03
################################################## 1154/1154
primary.xml.gz            100% |=========================| 168 kB    00:42
http://mirror.hiwaay.net/redhat/fedora/linux/extras/6/i386/repodata/primary.xml.gz:
[Errno 4] Socket Error: timed out
Trying other mirror.
primary.xml.gz            100% |=========================| 1.2 MB    00:01
################################################## 3742/3742
primary.xml.gz            100% |=========================| 185 kB    00:01
################################################## 960/960
Reading Local RPMDB

[and then it waits forever]

doesn't respond to kill, but does exit on kill -9

Version-Release number of selected component (if applicable):
3.0

How reproducible:
always

Steps to Reproduce:
1. su -
2. yum upgrade
  
Actual results:
hangs

Expected results:
packages upgraded

Additional info:
strace attached.
Comment 1 Greg Martyn 2006-10-26 22:20:31 EDT
Created attachment 139547 [details]
last 30 lines of strace
Comment 2 Chad Sellers 2006-10-30 20:40:22 EST
I'm seeing the same symptoms on an up to date FC6 machine as of the morning of
10/30/06. This is reproduceable with all rpm actions on my machine. For example,
rpm -qa hangs as well. Output follows:

# rpm -qa
tzdata-2006m-2.fc6
libICE-1.0.1-2.1

Then it just hangs. Last line of strace follows:
futex(0xb7d46fd0, FUTEX_WAIT, 2, NULL <unfinished ...>

Looks like it's never returning from the futex() call.
Comment 3 Chad Sellers 2006-10-31 11:58:13 EST
Note that this is solved by a reboot. I haven't been able to reproduce it since
I rebooted. Presumeably the futex in question was somehow never reset by a
previous process.
Comment 4 Greg Martyn 2006-10-31 12:30:56 EST
I can confirm that rebooting resolves this problem.
Comment 5 Greg Martyn 2006-11-01 23:35:39 EST
Ah, it's back:
# strace rpm -Uvh freshrpms-release-1.1-1.fc.noarch.rpm
...
getgid32()                              = 0
getuid32()                              = 0
stat64("/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/lib/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/lib/rpm", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
access("/var/lib/rpm", W_OK)            = 0
access("/var/lib/rpm/__db.001", F_OK)   = 0
access("/var/lib/rpm/Basenames", F_OK)  = 0
stat64("/var/lib/rpm/Basenames", {st_mode=S_IFREG|0644, 
st_size=10711040, ...}) = 0
open("/var/lib/rpm/Basenames", O_RDONLY|O_LARGEFILE) = 6
fcntl64(6, F_SETFD, FD_CLOEXEC)         = 0
read(6, "\0\0\0\0\1\0\0\0\0\0\0\0a\25\6\0\10\0\0\0\0\20\0\0\0\10"..., 512) = 
512
close(6)                                = 0
open("/var/lib/rpm/Basenames", O_RDONLY|O_LARGEFILE) = 6
fcntl64(6, F_SETFD, FD_CLOEXEC)         = 0
fstat64(6, {st_mode=S_IFREG|0644, st_size=10711040, ...}) = 0
stat64("/bin", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
futex(0xb7d9b520, FUTEX_WAIT, 2, NULL


I should note that this is a SMP computer.
Comment 6 Michael De La Rue 2006-11-03 02:15:41 EST
don't know if this is related, but here's a backtrace of a locked up yum on a
single processor computer.

From ps aux I was running

/usr/bin/python /usr/bin/yum -y install galeon gnash gnash-plugin darcs gnuchess
ncftp mediawiki igal

(gdb) bt
#0  0x001e97ee in __memp_fput_rpmdb () from /usr/lib/librpmdb-4.4.so
#1  0x0015fb17 in __ham_quick_delete_rpmdb () from /usr/lib/librpmdb-4.4.so
#2  0x001ab388 in __db_c_put_rpmdb () from /usr/lib/librpmdb-4.4.so
#3  0x001afcbf in __db_c_put_pp_rpmdb () from /usr/lib/librpmdb-4.4.so
#4  0x001408fd in tagType () from /usr/lib/librpmdb-4.4.so
#5  0x0013b2f0 in rpmdbAdd () from /usr/lib/librpmdb-4.4.so
#6  0x49aad385 in rpmpsmStage () from /usr/lib/librpm-4.4.so
#7  0x49aaebf2 in rpmpsmStage () from /usr/lib/librpm-4.4.so
#8  0x49aadfc2 in rpmpsmStage () from /usr/lib/librpm-4.4.so
#9  0x49aaebf2 in rpmpsmStage () from /usr/lib/librpm-4.4.so
#10 0x49aad9de in rpmpsmStage () from /usr/lib/librpm-4.4.so
#11 0x49ad4f66 in rpmtsRun () from /usr/lib/librpm-4.4.so
#12 0x00e9b56d in rpmts_Create () from
/usr/lib/python2.4/site-packages/rpm/_rpmmodule.so
#13 0x4ab9f51d in PyCFunction_Call () from /usr/lib/libpython2.4.so.1.0
#14 0x4abd9abd in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0
#15 0x4abdad68 in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0
#16 0x4abd9526 in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0
#17 0x4abd962f in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0
#18 0x4abdad68 in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0
#19 0x4abd9526 in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0
#20 0x4abdad68 in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0
#21 0x4abdadf3 in PyEval_EvalCode () from /usr/lib/libpython2.4.so.1.0
#22 0x4abf7a98 in Py_CompileString () from /usr/lib/libpython2.4.so.1.0
#23 0x4abf91a8 in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.4.so.1.0
#24 0x4abf988a in PyRun_AnyFileExFlags () from /usr/lib/libpython2.4.so.1.0
#25 0x4ac00285 in Py_Main () from /usr/lib/libpython2.4.so.1.0
#26 0x08048582 in main ()
Comment 8 Jeff Johnson 2006-11-03 23:33:53 EST
You have stale locks. Fix by doing (also done on reboot):
    rm -f /var/lib/rpm/__db*
Comment 9 Greg Martyn 2006-11-05 04:46:22 EST
Ok. A meaningful error message would be very helpful here. Should I open up a 
separate wishlist bug report for that?
Comment 10 Allen Chen 2006-11-06 22:21:42 EST
I also met this issue with Yum sometimes, it is Ok after reboot and can
reproduce with "yum update" or "yum install packages list".
After killed the hanged YUM, the rpm and yum command don't work right then
before reboot the system!

Platform: FC6 on IBM ThinkPad T23!
  
Comment 11 Hal Canary 2006-11-18 08:41:53 EST
I had yum hang during a 'yum update' as well.  I did a 'rm -f
/var/lib/rpm/__db*' and it fixed the problem.  Thank you.
Comment 12 Andreas M. Kirchwitz 2006-11-18 23:03:32 EST
This is the same problem as described in bug #206275 and bug #215901

I can verify this on a FC5 machine (rpm 4.4.2), too.
Usually, rpm hangs during the daily /etc/cron.daily/rpm job,
and as a result any further calls to yum/rpm won't work.

With "cd /var/lib/rpm && /usr/lib/rpm/rpmdb_stat -CA" you see
the locks, and "rm /var/lib/rpm/__db*" helps. But how can this
be avoided in the first place?

The bug occurs every two or three days. It's not only annoying,
but it's a high security risk because automatic installation of
security updates won't work.

First time I noticed the bug might be one or two months ago.
As it affects FC5 and FC6, the problem could be kernel-related
(a bug that was introduced in a specific kernel version).
Maybe 2.6.18 related. Never noticed this in the early days
of FC5 (2.6.15).
Comment 13 Pere Benavent 2006-11-30 05:17:50 EST
I've been suffering this problem as is described above. I've done that suggested
commands (#rm -f __db*) at /var/lib/rpm at the problem has gone away.

Since the problem has been solved, yum commands seems to take more time than
before. I'll keep on looking how it evolves.

Anyway, formerly, a "rpm --rebuilddb" has finish properly (very time consuming,
by the way), but any yum command finish in the right way.

My workstation runs a Fedora Core 6, which has been uptaded from Fedora Core 5
without problems.

(Please, excuse my english)
Comment 14 Brian Millett 2006-11-30 10:29:25 EST
fc6 dell d820n laptop.  
This has just happened to me also.  I had a problem with yum giving an error
about the database.  No I do not have that error.  I tried to run "rpm
--rebuilddb" and got this error:

[bpm]$ sudo rpm --rebuilddb
error: db4 error(-1076673940) from db->open: Unknown error: -1076673940
error: cannot open Packages index using db3 -  (-1076673940)

My question is why is it a db4 error using db3 ???

I rebooted and rpm / yum worked again.  This am I tried to do a yum update and I
got a segmentation fault.  Tried it a second time and it worked.

I'm just not feeling that rpm is very stable at this time.
Comment 15 Pere Benavent 2006-12-01 14:41:21 EST
Hi again, hope it would helps...


I type a "#yum update", I answer yes and everything run as is has to do it 'till
yum gets stucked. I look for yum's PID, then I type and '#kill -18
$whatever-PID' and yum download restart and finish properly.

I may guarantee you there where no network problems, meanwhile I was browsing as
I  allways do it.

I'll keep you informed.

Pere
Comment 16 Jeff Johnson 2006-12-03 08:00:10 EST
Re comment 14: the "db3" in the error message is the api index, not the version number as in db-4.x.y.
Yes, "db3" is expected, Berkely DB still uses the 3rd API for Berkeley DB even though the version is now 4.
Comment 17 Jeff Johnson 2006-12-03 13:16:24 EST
FYI: Most rpmdb "hangs" are now definitely fixed by purging stale read locks when opening
a database environment in rpm-4.4.8-0.4. There's more todo, but I'm quite sure that a
large class of problems with symptoms of "hang" are now corrected.

UPSTREAM
Comment 18 Jeremy Katz 2007-01-03 13:05:14 EST
*** Bug 221044 has been marked as a duplicate of this bug. ***
Comment 19 Jeremy Katz 2007-01-03 13:05:19 EST
*** Bug 220685 has been marked as a duplicate of this bug. ***
Comment 20 Jeremy Katz 2007-01-03 13:05:36 EST
*** Bug 214129 has been marked as a duplicate of this bug. ***
Comment 21 Jeremy Katz 2007-01-03 13:05:56 EST
*** Bug 220987 has been marked as a duplicate of this bug. ***
Comment 22 Jeremy Katz 2007-01-03 13:06:06 EST
*** Bug 220988 has been marked as a duplicate of this bug. ***
Comment 23 Jeremy Katz 2007-01-03 13:22:05 EST
*** Bug 221053 has been marked as a duplicate of this bug. ***
Comment 24 Mukul Agarwal 2007-01-04 06:31:27 EST
the rebooting of the system does not solve the problem. May be re-installation
of the effected .rpms will be cure the problem. request a listy of the effected
rpms.
Comment 25 Bryan Christ 2007-01-19 11:22:14 EST
I am also having this problem on FC6.  After doing a "rm -f /var/lib/rpm/__db*"
yum no longer hangs.  Anyone know what the root cause of the problem is?
Comment 26 Bryan Christ 2007-01-19 11:25:40 EST
katzj:  I noticed that bug 211786 appears to describe the same problem with the
same workaround.
Comment 27 Panu Matilainen 2007-07-17 08:40:13 EDT

*** This bug has been marked as a duplicate of 213963 ***

Note You need to log in before you can comment on or make changes to this bug.