Bug 443373 - rpmdb: write: 0x3896790, 16384: No space left on device
rpmdb: write: 0x3896790, 16384: No space left on device
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: anaconda (Show other bugs)
4.7
ia64 Linux
low Severity high
: beta
: ---
Assigned To: Anaconda Maintenance Team
:
: 443673 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-04-21 04:54 EDT by Alexander Todorov
Modified: 2008-07-24 15:06 EDT (History)
7 users (show)

See Also:
Fixed In Version: RHBA-2008-0653
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-07-24 15:06:20 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
output of df after stage2 is loaded (7.10 KB, image/png)
2008-04-23 05:08 EDT, Alexander Todorov
no flags Details
output of df after the error occured (7.11 KB, image/png)
2008-04-23 05:11 EDT, Alexander Todorov
no flags Details
Increase the fs size. (486 bytes, patch)
2008-04-23 07:48 EDT, Joel Andres Granados
no flags Details | Diff

  None (edit)
Description Alexander Todorov 2008-04-21 04:54:35 EDT
Description of problem:
Installation fails with the message in subject just after (or while) checking
dependencies.

Version-Release number of selected component (if applicable):
anaconda-10.1.1.86-1

How reproducible:


Steps to Reproduce:
1. Install RHEL4-U7 tree from 20080418
2. 
3.
  
Actual results:
+--------+ Dependency Check +---------+                     
|                                     |                     
| Checking dependencies in packages   |                     
| selected for installation...rpmdb: write: 0x3896790, 16384: No space left on
device 
|                                     |


Expected results:
installation proceeds without errors

Additional info:
Comment 5 Alexander Todorov 2008-04-21 07:34:04 EDT
The error messages from other executions:
rpmdb: write: 0x319d740, 16384: No space left on device
rpmdb: write: 0x32163d0, 16384: No space left on device

the first hex number is different but the second one is the same as in comment #0.

Comment 6 Alexander Todorov 2008-04-21 08:11:18 EDT
We have ramdisk_size=12288 argument for all machines that failed. Is this
something that may be causing the problem?
Comment 7 Panu Matilainen 2008-04-21 10:31:47 EDT
IIRC in RHEL 4, anaconda uses rpm's dependency resolution mechanism and rpm
caches the dependency lookups in BDB, if that goes to a ramdisk... It's by no
means new behavior, but maybe the ramdisk contents are now just that little bit
closer to  max than it was before. So yeah, I suppose increasing ramdisk_size
should help.
Comment 8 Alexander Todorov 2008-04-21 12:11:20 EDT
Panu, 
see comment #6 for the actual ramdisk_size value but I was told by anaconda
folks that this parameter is not taken into account anymore.
Comment 9 Alexander Todorov 2008-04-22 10:29:15 EDT
(In reply to comment #7)
> IIRC in RHEL 4, anaconda uses rpm's dependency resolution mechanism and rpm
> caches the dependency lookups in BDB, if that goes to a ramdisk... It's by no
> means new behavior, but maybe the ramdisk contents are now just that little bit
> closer to  max than it was before. So yeah, I suppose increasing ramdisk_size
> should help.

Panu,
even with ramdisk_size=50000 I can still see the error. 
Comment 10 David Lehman 2008-04-22 16:06:39 EDT
*** Bug 443673 has been marked as a duplicate of this bug. ***
Comment 11 David Lehman 2008-04-22 19:21:26 EDT
I got nowhere on this today. Can someone see what the output of 'df' looks like
on these systems both before and, if possible, after the error? anaconda.log and
syslog might also be of some use.

Also, can someone give a little more information about ramdisk_size getting
ignored? Specifically, what does decide the size of the ramdisk? A quick look at
an i386 system seems to support the notion that the ramdisk_size parameter does
nothing (6M rootfs with ramdisk_size=8192 on cmdline).
Comment 12 Panu Matilainen 2008-04-23 03:15:23 EDT
Just a few datapoints from rpm POV:
- like said, rpm caches dependency lookup results on disk (Berkeley DB) if
possible, that's why it would be writing to disk at that point
- the caching has been there for a long time, certainly in all RHEL 4 versions
- I looked through recent anaconda changelogs and didn't see anything obvious
that would affect this, and the fact it only happens on ia64 tells it's not a
very fundamental change in anaconda, otherwise we'd see this on all platforms
- if it comes to that, it is possible to disable the caching in rpm (a one-liner
patch) at a slight performace cost, but that'd be treating the symptoms instead
of the disease
Comment 13 Alexander Todorov 2008-04-23 03:53:09 EDT
(In reply to comment #11)
> I got nowhere on this today. Can someone see what the output of 'df' looks like
> on these systems both before and, if possible, after the error? anaconda.log and
> syslog might also be of some use.
> 

anaconda.log and syslog are available in any of the links in comment #4 (look
for "anamon" link)

I can provide `df' just after stage2 has started. Is this what you refer to
"before"? Unfortunately after the error it's not possible to switch to other tty
and getting `df' will be hard I think. jgranado is trying to get more info on
that using pdb. 


> Also, can someone give a little more information about ramdisk_size getting
> ignored? Specifically, what does decide the size of the ramdisk? A quick look at
> an i386 system seems to support the notion that the ramdisk_size parameter does
> nothing (6M rootfs with ramdisk_size=8192 on cmdline).

Comment 14 Alexander Todorov 2008-04-23 05:08:34 EDT
Created attachment 303459 [details]
output of df after stage2 is loaded
Comment 15 Alexander Todorov 2008-04-23 05:11:26 EDT
Created attachment 303460 [details]
output of df after the error occured

Both outputs show that space is used at 100%. The only difference is the
numbers:

Before:
Filesystem     Used
rootfs	       10656
/dev/root.old  10656

After:
Filesystem     Used
rootfs	       10728
/dev/root.old  10728

everything else is the same
Comment 16 Joel Andres Granados 2008-04-23 07:48:38 EDT
Created attachment 303482 [details]
Increase the fs size.

Ok, I think I found the reason for this issue.
We just don't have enough ramdisk.  the ramdisk_size option works fine, we just
dont make a good use of it.  The ramdisk grows, what does not grow is the
filesystem that we use.  It needs to be bigger.
The reason for this issue is most likely something new that uses the ram disk
and we didn't know about it.  What needs to be done is to increase the
filesystem size.
Question is.  Will this increase be enough?
Comment 17 Alexander Todorov 2008-04-24 04:53:06 EDT
(In reply to comment #16)
> Question is.  Will this increase be enough?

Testing with initrd.img provided by Joel:
* default package set install completed
* everything package set completed

I didn't observe the rpmdb error and was able to reboot into the newly installed
system. 

Please commit the fix and build a new package ASAP.
Comment 18 David Lehman 2008-04-24 12:22:21 EDT
Fixed in anaconda-10.1.1.88-1.
Comment 22 errata-xmlrpc 2008-07-24 15:06:20 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2008-0653.html

Note You need to log in before you can comment on or make changes to this bug.