Bug 812351 - mke2fs on 15TB volume requires 728084kB mem, causing OOM
mke2fs on 15TB volume requires 728084kB mem, causing OOM
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: e2fsprogs (Show other bugs)
6.3
x86_64 Unspecified
urgent Severity urgent
: rc
: ---
Assigned To: Eric Sandeen
BaseOS QE - Apps
abrt_hash:09c548cdca9f8301bb598222638...
:
Depends On:
Blocks: 760559 767727 840699
  Show dependency treegraph
 
Reported: 2012-04-13 09:29 EDT by Alexander Todorov
Modified: 2012-07-31 01:53 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-07-28 08:23:24 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
File: anaconda-tb-3oFRBC (235.18 KB, text/plain)
2012-04-13 09:29 EDT, Alexander Todorov
no flags Details

  None (edit)
Description Alexander Todorov 2012-04-13 09:29:53 EDT
libreport version: 2.0.9
cmdline:        /usr/bin/python  /usr/bin/anaconda
executable:     /usr/bin/anaconda
exnFileName:    /tmp/anaconda-tb-3oFRBC
hashmarkername: anaconda
kernel:         2.6.32-262.el6.x86_64
other involved packages: 
product:        Red Hat Enterprise Linux
reason:         FormatCreateError: ('format failed: 1', '/dev/mapper/vg00-lv_root')
release:        Cannot get release name.
time:           Fri 13 Apr 2012 01:28:40 PM EDT
version:        6.3

anaconda-tb-3oFRBC: Text file, 240823 bytes

description:
:The following was filed automatically by anaconda:
:anaconda 13.21.167 exception report
:Traceback (most recent call first):
:  File "/usr/lib/anaconda/storage/formats/fs.py", line 372, in doFormat
:    raise FormatCreateError("format failed: %s" % rc, self.device)
:  File "/usr/lib/anaconda/storage/formats/fs.py", line 964, in doFormat
:    FS.doFormat(self, *args, **kwargs)
:  File "/usr/lib/anaconda/storage/formats/fs.py", line 878, in create
:    return self.doFormat(*args, **kwargs)
:  File "/usr/lib/anaconda/storage/deviceaction.py", line 290, in execute
:    options=self.device.formatArgs)
:  File "/usr/lib/anaconda/storage/devicetree.py", line 713, in processActions
:    action.execute(intf=self.intf)
:  File "/usr/lib/anaconda/storage/__init__.py", line 349, in doIt
:    self.devicetree.processActions()
:  File "/usr/lib/anaconda/packages.py", line 110, in turnOnFilesystems
:    anaconda.id.storage.doIt()
:  File "/usr/lib/anaconda/dispatch.py", line 210, in moveStep
:    rc = stepFunc(self.anaconda)
:  File "/usr/lib/anaconda/dispatch.py", line 126, in gotoNext
:    self.moveStep()
:  File "/usr/lib/anaconda/dispatch.py", line 233, in currentStep
:    self.gotoNext()
:  File "/usr/lib/anaconda/gui.py", line 1421, in setScreen
:    (step, anaconda) = self.anaconda.dispatch.currentStep()
:  File "/usr/lib/anaconda/gui.py", line 1614, in setup_window
:    self.setScreen()
:  File "/usr/lib/anaconda/gui.py", line 1627, in run
:    self.setup_window()
:  File "/usr/lib/anaconda/gui.py", line 1334, in run
:    self.icw.run ()
:  File "/usr/bin/anaconda", line 1127, in <module>
:    anaconda.intf.run(anaconda)
:FormatCreateError: ('format failed: 1', '/dev/mapper/vg00-lv_root')

environ:
:LC_NUMERIC=C
:LANGKEY=en
:LVM_SUPPRESS_FD_WARNINGS=1
:GCONF_GLOBAL_LOCKS=1
:PATH=/tmp/product:/tmp/updates:/usr/bin:/bin:/sbin:/usr/sbin:/mnt/sysimage/bin:/mnt/sysimage/usr/bin:/mnt/sysimage/usr/sbin:/mnt/sysimage/sbin:/mnt/sysimage/usr/X11R6/bin:/tmp/DD/bin:/tmp/DD/sbin:/tmp/DD/usr/bin:/tmp/DD/usr/bin:/sbin:/usr/sbin
:PIXMAPPATH=/tmp/updates/pixmaps/:/tmp/updates/:/tmp/product/pixmaps/:/tmp/product/:pixmaps/:/usr/share/anaconda/pixmaps/:/usr/share/pixmaps/:/usr/share/anaconda/:/usr/share/python-meh/
:HOME=/tmp
:DISPLAY=:1
:LANG=en_US.UTF-8
:TERM=linux
:TZ=America/New_York
:XAUTHORITY=//.Xauthority
:GLADEPATH=/tmp/updates/:/tmp/updates/ui/:ui/:/usr/share/anaconda/ui/:/usr/share/python-meh/
:LIBUSER_CONF=/tmp/libuser.9Wfyv2
:LD_LIBRARY_PATH=/tmp/product:/tmp/updates:/lib64:/usr/lib64:/lib:/usr/lib:/tmp/DD/lib64:/tmp/DD/usr/lib64:/tmp/DD/lib:/tmp/DD/usr/lib
:GNOME_DISABLE_CRASH_DIALOG=1
:TERMINFO=/etc/linux-terminfo
:PYTHONPATH=/tmp/product:/tmp/updates/textw:/tmp/updates/iw:/tmp/updates:/tmp/updates
:LINGUAS=English
:DEBUG=
:LC_COLLATE=C
:MALLOC_PERTURB_=204
:MALLOC_CHECK_=2
Comment 1 Alexander Todorov 2012-04-13 09:29:55 EDT
Created attachment 577338 [details]
File: anaconda-tb-3oFRBC
Comment 2 Alexander Todorov 2012-04-13 09:31:43 EDT
Reproduced during testing another bug. See steps to reproduce:
https://bugzilla.redhat.com/show_bug.cgi?id=767727#c5
Comment 3 Jesse Keating 2012-04-17 17:10:46 EDT
Quoting the other bug:

Tested with ks.cfg configuration from comment #0 and two sparse disk images.
Each sized 16492674416641 bytes.

# dd if=/dev/zero of=large2.img seek=15T count=1 bs=1


I hit bug 812351.
Comment 4 David Cantrell 2012-05-24 15:54:45 EDT
Is this still happening on the latest 6.3 nightly trees?
Comment 5 Alexander Todorov 2012-05-25 04:02:22 EDT
(In reply to comment #4)
> Is this still happening on the latest 6.3 nightly trees?

Yes,
I tested with snapshot #5 which is the most recent working tree I have ATM.
Comment 6 David Lehman 2012-05-29 13:47:16 EDT
17:28:14,436 ERR kernel:Out of memory: Kill process 1409 (mke2fs) score 565 or sacrifice child
17:28:14,436 ERR kernel:Killed process 1409, UID 0, (mke2fs) total-vm:728084kB, anon-rss:606672kB, file-rss:8kB
Comment 7 Alexander Todorov 2012-05-31 04:10:59 EDT
13:27:34,596 INFO    : Detected 1008M of memory
13:27:34,596 INFO    : Swap attempt of 1008M to 1008M

Isn't that not enough? I also have swap space available. 

David,
is it expected that mke2fs consumes that much memory when creating a large filesystem ? iow is there a memory leak in mke2fs? OTOH shouldn't we make use of the available swap space?
Comment 8 David Lehman 2012-06-05 12:55:08 EDT
(In reply to comment #7)
> 13:27:34,596 INFO    : Detected 1008M of memory
> 13:27:34,596 INFO    : Swap attempt of 1008M to 1008M
> 
> Isn't that not enough? I also have swap space available. 
> 
> David,
> is it expected that mke2fs consumes that much memory when creating a large
> filesystem ? iow is there a memory leak in mke2fs?

That sure looks like a lot of memory to me.

> OTOH shouldn't we make use of the available swap space?

Activating swap early is non-trivial in many cases as it requires enforcing an artificial ordering constraint on the storage creation/activation operations.
Comment 9 David Lehman 2012-06-05 13:25:11 EDT
Reassigning to e2fsprogs as anaconda cannot do anything about how much memory the tools require to create huge filesystems. If there's no bug there, maybe a release note would be appropriate.
Comment 10 Ric Wheeler 2012-07-28 08:23:24 EDT
You need to size your memory to the size of the file system you need to create, we don't have current plans to fix this so I will close out this BZ.  Note that other file systems (XFS or btrfs) will do better here.

Note You need to log in before you can comment on or make changes to this bug.