Bug 1328151 - Disk space requirements are too conservative
Summary: Disk space requirements are too conservative
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: 24
Hardware: All
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Jiri Konecny
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords: Reopened
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-18 15:14 UTC by Orion Poplawski
Modified: 2017-07-14 07:16 UTC (History)
7 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2016-11-15 13:32:55 UTC


Attachments (Terms of Use)
anaconda.log (34.30 KB, text/plain)
2016-10-11 16:01 UTC, Orion Poplawski
no flags Details
packaging.log (26.57 KB, text/plain)
2016-10-11 16:01 UTC, Orion Poplawski
no flags Details
kickstart (9.76 KB, text/plain)
2016-10-11 16:03 UTC, Orion Poplawski
no flags Details
packaging.log (31.14 KB, text/plain)
2016-10-21 16:27 UTC, Orion Poplawski
no flags Details

Description Orion Poplawski 2016-04-18 15:14:59 UTC
Description of problem:

Installing F24 desktop system I got:

21:40:37,539 INFO packaging: 4030 packages selected totalling 18.02 GiB

While it looks like the maximum total installed system use during install was 15GB including the 3.8GB for /dnf.package.cache.  So once the dnf.package.cache was removed after install, the installed system used around 11GB, which I then bumped up to 12GB with other post install items.

Beyond a reasonable headroom (? but 3GB/20% seems excessive to me) to account for errors in the rpm estimates so that the install completes properly, I don't think the installer has any role in enforcing a minimum size of /.  With the removal on /dnf.package.cache there will be a significant amount of free space post-install.

Version-Release number of selected component (if applicable):
24.13.3-1

Comment 1 Jiri Konecny 2016-04-19 12:29:50 UTC
Hello Orion,

you are right, Anaconda shouldn't enforce the minimum size of /. Bad thing is we need to. We can't just launch the installation then let the DNF download packages and wait for the crash because DNF will start checks after downloading not before.

And I said in bug #1327774 we don't have all the information to create precise space check and the old one crashed when user did minimal install with really small amount of space in /.

However I may have the solution for you. We can create option to Kickstart %packages or Boot option to disable space check at all. This mean that your installation will crash on DNF or RPM space check in the installation but not on our space check.

Do you like this solution?

Jirka

Comment 2 Jiri Konecny 2016-07-22 11:31:46 UTC
It should be fixed by this:

https://github.com/rhinstaller/anaconda/commit/5697615a7518bba5ad46d015703cf7367fdc3594

in Fedora 25.

Comment 3 Orion Poplawski 2016-09-06 16:47:28 UTC
I'm not sure this is working right.  First - is there a flag that I need to set?  I can't figure that out form the commit.

Second, I'm seeing:

16:33:24,753 INFO packaging: Sufficient mountpoints found: {}
16:33:24,753 INFO packaging: 4141 packages selected totalling 18.88 GiB
16:33:27,310 INFO packaging: Sufficient mountpoints found: {}
16:33:27,311 ERR packaging: Installation failed: PayloadError('Not enough disk space to download the packages.',)

but without the nokill flag the system immediately shuts down.  Seems like the message should be displayed.  This is a kickstart text mode install.

This is with 25.20-1.

Comment 4 Jiri Konecny 2016-09-23 12:36:13 UTC
Hello Orion,

can you please post logs and your kickstart here? 
Did you got message "Enter 'b' to ignore the warning and attempt to install anyway" in the installation or the installation failed without interaction?

There is no required flag to set. You only need to start kickstart automatic installation and confirm that you want to start installation even when you don't have enough space.

Jirka K.

Comment 5 Orion Poplawski 2016-10-11 16:01 UTC
Created attachment 1209200 [details]
anaconda.log

No prompt - simply "anaconda halting due to nokill flag."

Comment 6 Orion Poplawski 2016-10-11 16:01 UTC
Created attachment 1209201 [details]
packaging.log

Comment 7 Orion Poplawski 2016-10-11 16:03 UTC
Created attachment 1209202 [details]
kickstart

Comment 8 Jiri Konecny 2016-10-21 10:54:38 UTC
Hello Orion,

thank you for the logs. I don't know why but I'm missing DEBUG messages in packaging.log file so I can't tell what is happening there exactly. There are only DEBUG messages from DNF.

Could you please run the installation again with this updates image[1] and give me packaging.log again. I can't reproduce this on my side.

Could you confirm that in the machine is just one 15GB disk for an installation?

Thank you,
Jirka


[1] https://jkonecny.fedorapeople.org/redhat/f25-improve-logging.img

Comment 9 Orion Poplawski 2016-10-21 16:27 UTC
Created attachment 1212900 [details]
packaging.log

Comment 10 Orion Poplawski 2016-10-21 16:28:51 UTC
(In reply to Jiri Konecny from comment #8)
> Could you confirm that in the machine is just one 15GB disk for an
> installation?

Yup - VM install into 15GB volume.

Comment 11 Jiri Konecny 2016-10-25 07:22:50 UTC
I think I have found the root case. There was an error in algorithm for picking download location.

PR: https://github.com/rhinstaller/anaconda/pull/844

If you want to test it now, you can use this updates image:

https://jkonecny.fedorapeople.org/redhat/f25_fix_bug_1328151.img


Jirka

Comment 12 Orion Poplawski 2016-10-25 22:34:49 UTC
okay, with that I now get to:

Error Summary
-------------
Disk Requirements:
   At lest 5626MB more space needed on the / filesystem.



What do you want to do now?
1) Report Bug
2) Debug
3) Run shell
4) Quit

Please make your choice from above:



Which seems reasonable, although I doubt you want any more bug reports :).


Now if there was only a way to not have the dnf.package.cache on /mnt/sysimage...


Thanks.

Comment 13 Jiri Konecny 2016-10-26 07:21:58 UTC
This error is directly from DNF and I can't help you much with it. 

About the dnf.package.cache, you can move it on an another place, more precise it will try to do that automatically. When the packages can't be downloaded and installed to the same place (your situation) then it will try to take the biggest mountpoint large enough for the download from the places below and if it is possible do not take '/' mountpoint.

Possible locations are:

'/tmp'
'/'
'/var/tmp'
'/mnt/sysimage'
'/mnt/sysimage/home'
'/mnt/sysimage/tmp'
'/mnt/sysimage/var'

In other words, if you want the dnf.package.cache elsewhere you can create for example /mnt/sysimage/home (/home in kickstart part command) with the size enough for downloading (about 5GB for you) and it will be placed there.

Thank you for your cooperation,
Jirka

Comment 14 Fedora Update System 2016-10-31 10:21:57 UTC
anaconda-25.20.6-1.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-98dad065d7

Comment 15 Fedora Update System 2016-11-03 18:23:03 UTC
anaconda-25.20.6-1.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2016-11-07 23:31:17 UTC
anaconda-25.20.7-1.fc25 python-blivet-2.1.6-3.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-91844f4982

Comment 17 Fedora Update System 2016-11-09 02:26:14 UTC
anaconda-25.20.8-1.fc25, python-blivet-2.1.6-3.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-91844f4982

Comment 18 Fedora Update System 2016-11-15 13:32:55 UTC
anaconda-25.20.8-1.fc25, python-blivet-2.1.6-3.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Bruce Jerrick 2016-12-04 04:43:49 UTC
I have had a similar problem for some time -- anaconda would silently
crash back to a boot menu if the package selection was large.  My workaround
was to add this to DOWNLOAD_MPOINTS in dnfpayload.py:

  /mnt/sysimage/usr

That fixes it for those weird people who use a separate /usr filesystem
(resulting in a small / (root)), along with a non-large /var and /home, and
not a lot of memory (non-large /tmp).

I have not yet tested the new anaconda and python-blivet without this
workaround.

In any case, it seems reasonable to include /mnt/sysimage/usr in the list
of candidates.

Comment 20 Jiri Konecny 2016-12-05 07:42:20 UTC
Hello Bruce,

Thank you for your info, it could be helpful for someone. 
I would like to see this fixed to have really good required space checker. The problem is, we don't have much info about where the package requires how much space (we hardly know how much space the package will take in overall) and the partitioning can be done in almost infinite possible ways. I'm still trying to find some way to solve this and I will continue.

Jirka

Comment 21 Bruce Jerrick 2016-12-05 18:08:01 UTC
Jiri:
Something screwy is going on.  Last July, my /mnt/sysimage/usr addition worked
even with a small amount of memory (this is on a VMware virtual machine).
But now it doesn't -- I need 12 GiB of memory, else the anaconda crash and
reboot.  (That 12 GiB allows a 6 GiB /tmp .)
Until I get this sorted out, giving all the details would probably just
confuse things.  But there is this: In inst.text mode, there is a report:

  "Not enough space in file systems for the current software selection.
   An additional 27.95 GiB is needed."

That's for 4651 packages, 4698 MiB total size of the packages (the .rpm files),
  resulting in ~16.2 GiB total disk usage in the finished system.

Comment 22 Bruce Jerrick 2017-07-14 00:27:56 UTC
I still get the crashing I reported on Comment 19, now when trying an F26 Release
installation (anaconda 26.21.11-1.fc26).  Can't /mnt/sysimage/usr be added to to DOWNLOAD_MPOINTS in dnfpayload.py ?  Having a (large) separate /usr and a
relatively small everything else seems like a reasonable configuration.

Comment 23 Jiri Konecny 2017-07-14 07:16:25 UTC
Hi Bruce,

Could you please file a new bug for this.

I'm thinking about adding /mnt/sysimage/usr to the DOWNLOAD_MPOINTS. The problem is that you don't know how much space is required for installed files specifically in /usr. In other words, you aren't able to find out if the installation will be successful. 

I see this option here:

Use /mnt/sysimage/usr only if user have enough space there for download + installation. In other words you would need 28GB partition size for your future /usr.


Would that be enough for your use-case?


Note You need to log in before you can comment on or make changes to this bug.