Bug 873831

Summary: Make /tmp bigger, or use target image for yum downloads
Product: [Fedora] Fedora Reporter: Orion Poplawski <orion>
Component: anacondaAssignee: Will Woods <wwoods>
Status: ASSIGNED --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: amcnabb, awilliam, ddumas, edgar.hoch, g.kaviyarasu, jonathan, larryoleary, loleary, mattdm, mkolman, redhat-bugzilla, robatino, vanmeeuwen+fedora
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: RejectedBlocker
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Orion Poplawski 2012-11-06 19:28:15 UTC
Description of problem:

On anaconda install system, /tmp is the default size of RAM/2.  Trying to install to a VM with 1GB RAM (and 498MB /tmp) I ran out of space in /tmp to download the yum repositories (I enable a lot of repos via cobber during install).

Also note that the traceback generated by this:

yum.Errors.RepoError: Insufficient space in download directory /tmp/yum.cache/fedora-18-devel-x86_64

was not logged anywhere that I could find in /tmp although there was still some space there.

Version-Release number of selected component (if applicable):
18.24

Comment 1 Orion Poplawski 2012-11-06 23:16:22 UTC
Didn't these files used to go into the install image?  I could see not wanting them there after the install, but we generally have more space there.

Comment 2 Orion Poplawski 2012-12-03 22:23:21 UTC
Proposing this as a blocker.  I'm not sure if there is a direct criteria for this, but I suspect that this will bump up the RAM requirements for a system install.

Comment 3 Adam Williamson 2012-12-05 18:17:04 UTC
Discussed at 2012-12-05 blocker review meeting - http://meetbot.fedoraproject.org/fedora-bugzappers/2012-12-05/f18final-blocker-review-2.2012-12-05-17.01.log.txt . Based on our understanding of the issue this seems too much of a corner case - low RAM plus many external repositories - to block release. It is rejected as a blocker. If you believe the issue is more general and we misread it (please check the full IRC log for the full context of our discussion), please re-propose with a more detailed explanation.

We're not entirely sure if this is about repos used during install or repos set up in kickstart %post for the installed system, or what. It may help to have a bit more detail on the problem.

Comment 4 Orion Poplawski 2012-12-05 23:46:31 UTC
Ran a test in a 768MB VM (minimum listed graphical install memory requirements for F17) with:

url --url=http://fedora.cora.nwra.com/releases/test/18-Beta/Fedora/x86_64/os
repo --name=fedora-18-updates-x86_64 --baseurl=http://fedora.cora.nwra.com/updates/18/x86_64 
repo --name=fedora-18-updates-testing-x86_64 --baseurl=http://fedora.cora.nwra.com/updates/testing/18/x86_64 
repo --name=fedora-18-devel-x86_64 --baseurl=http://fedora.cora.nwra.com/development/18/x86_64/os 

%packages
@kde-desktop
%end

So this is not barebones, but is close to the "Fedora" + "Everything" + "Updates" mode that I think is fairly common.

This results in a 372MB /tmp and about 224MB+<largest package size> used for /tmp/yum.cache.  So yeah, this seems to be a problem only with lots of repos, or perhaps when yum needs to download filelists (which it didn't need to do in this case, but does in my usual install).

I still think it would be a good idea to mount /tmp with size=100% (or 90%).  Although a workaround is:

%pre
mount -o remount,size=100% /tmp

Comment 5 Andrew McNabb 2013-01-17 18:22:07 UTC
I am experiencing this problem on a virtual machine with 1024 MB of RAM. This amount of RAM meets the Fedora installation requirements: "At least 768 MB memory (RAM), 1 GB recommended for best performance" at fedoraproject.org/get-fedora. 
The error message does not give any indication that this is a problem with the amount of RAM:

"""
Insufficient space in download directory /tmp/yum.cache/anaconda/headers
   * free   92 k
   * needed 1.4 M
"""

I was using the Everything and Updates repositories but not any third-party repositories, so I don't think this should qualify as a corner case. It's too late for my vote to count, but I think this would have been a good Fedora 18 blocker. ;-)

Comment 6 Adam Williamson 2013-01-24 23:58:25 UTC
You must have been doing something else odd, because that's not typical. I did all my F18 validation testing in a VM with 1GB of RAM.

Comment 7 Andrew McNabb 2013-01-25 00:32:56 UTC
(In reply to comment #6)
> You must have been doing something else odd, because that's not typical. I
> did all my F18 validation testing in a VM with 1GB of RAM.

You may not have been installing very many packages. I didn't hit it when I only did the default packages.

Comment 8 Andrew McNabb 2013-04-23 21:18:09 UTC
Today I hit this bug on a computer (not a virtual machine) with 1 GB of RAM while trying to install Fedora 18.

If this comes up again as a potential blocker for Fedora 19 (assuming it's not fixed), it's worth noting that it's not quite the corner case that it was earlier understood to be. It occurs with RAM as high as 1 GB and without many (or even any?) packages from third-party repositories.

Comment 9 Larry O'Leary 2013-08-03 03:30:40 UTC
Same issue here with Fedora 19. Can't install it on a desktop with 1GB RAM even though it has 800GB disk space. 

Can't believe that this has gone unfixed for so many releases. 

I suggest that for QE we do a bit more qualification on the minimum requirements with a fully functioning system. This should include running with a many of the optional repos that are commonly used.

There wouldn't happen to be an anaconda parameter that can be passed to increase this space or put it on the install destination?

Comment 10 Fedora End Of Life 2013-12-21 09:18:00 UTC
This message is a reminder that Fedora 18 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 18. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '18'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 18's end of life.

Thank you for reporting this issue and we are sorry that we may not be 
able to fix it before Fedora 18 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior to Fedora 18's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 11 Adam Williamson 2013-12-21 19:29:56 UTC
c#9 says it's valid at least as recently as 19. probably valid still; let's just bump it to rawhide and call it a FutureFeature.

Comment 12 Larry O'Leary 2013-12-22 19:52:35 UTC
I can also confirm that this is still an issue in F20. I have to insert 2 1GB sticks of RAM to install if I want to use Fedora repos + RPM Fusion.

Considering this happens in very basic situations, it seems that until this is fixed, Fedora's minimum memory requirement should be stated at 1.5GB or more. It is obvious that 1GB won't work if the user adds sofware sources at install time.

To make matters worse, the install just hangs with no indication of the problem. From the user point-of-view, the installer is just taking days to setup software sources but a closer look reveals that the install was actually halted due to insufficient disk space -- even though there is plenty of free space.

Comment 13 Adam Williamson 2013-12-23 02:14:27 UTC
"Considering this happens in very basic situations, it seems that until this is fixed, Fedora's minimum memory requirement should be stated at 1.5GB or more."

It's a *minimum* requirement. You are not describing anything that could be considered a 'minimum' install; third-party repos are 'third-party' for a reason.

You can install F20 with 1GB of RAM. Heck, you can install it with 512MB. "Minimum" means "minimum"; it should be pretty obvious it doesn't mean "every possible install will work with this".

Comment 14 Adam Williamson 2013-12-23 02:15:37 UTC
"To make matters worse, the install just hangs with no indication of the problem."

This is what tends to happen with RAM exhaustion. It's very difficult to make something fail more gracefully in the case of RAM exhaustion without also making it fail needlessly in cases where it would have been able to succeed without your clever-clever failure case code.

Comment 15 Larry O'Leary 2013-12-23 02:42:15 UTC
Adam, I think are missing the bug here.

First off, this issue occurs when the package list is @core only. Very basic. Not a corner case at all. That seems to be pretty minimal to me.

Second, the issue isn't that we are actually running out of memory. We are running out of disk space on a tmp file system. This is a detectable issue and can easily be a avoided.

Comment 16 Adam Williamson 2013-12-23 03:09:58 UTC
It's 'disk space' that is actually RAM. anaconda cannot use space on any actual disk until you complete partitioning, because it doesn't know what disk space you want it to use and what disk space you don't. /tmp of the installation environment is not backed by any actual disk.

the selected package set is not important, what's important is the amount of repositories you enable and the amount of metadata required to configure those repos. if there's more metadata than space in /tmp - which is just part of your RAM - then you hit this bug.

Comment 17 Adam Williamson 2013-12-23 03:23:43 UTC
having said that, RAM wouldn't usually be absolutely exhausted in this case, so in theory I guess it should be possible to catch it and fail more nicely. failing nicely in cases where failing is the only thing we can do, and you're already trying something quite marginal, isn't very high up the anaconda development priority list though, I don't think, so I wouldn't count on it happening immediately. that's just my read of the situation.

Comment 18 Orion Poplawski 2013-12-23 03:28:48 UTC
So, is there any reason not to make /tmp 100% of RAM?  I've not had trouble with it since making it part of my standard kickstart.

Comment 19 Larry O'Leary 2013-12-23 05:47:52 UTC
(In reply to Adam Williamson from comment #16)
> It's 'disk space' that is actually RAM. anaconda cannot use space on any
> actual disk until you complete partitioning, because it doesn't know what
> disk space you want it to use and what disk space you don't. /tmp of the
> installation environment is not backed by any actual disk.

Disk partitioning and formatting has already been done at this point. It would be just as easy to use /tmp on the installation target.

Comment 20 Adam Williamson 2013-12-23 06:13:08 UTC
anaconda starts configuring repos as soon as it initializes the installation source spoke.

Comment 21 Larry O'Leary 2013-12-23 15:03:32 UTC
But it doesn't appear to download the repo data until the "Starting package installation" stage. That is when the no space left on device error occurs. 

Is that not the case?

Comment 22 Peter Lawler 2015-03-07 21:48:34 UTC
For the record, still happening in F21.

I've got my kickstart set to more or less all package groups to be installed, including --optional. I've also gone through and assembled a list of every -devel package, removed the conflicts, and added it as an %include.

Comment 23 Fedora Update System 2017-04-28 12:49:44 UTC
anaconda-26.21.4-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-f620d8c7c2

Comment 24 Adam Williamson 2019-12-11 23:43:06 UTC
I don't think this is actually fixed by 26.21.4-1, or has been fixed at all. Happen to be running an install ATM, and repodata still appears to be stored in the install environment's /tmp/dnf.cache .

For a Fedora 30 standard network install, it seems to be using about 272M.