Bug 1577041

Hi,
it is sort of tricky to calculate a disk space needed for a system-upgrade. The command is reporting only amount needed for downloaded packages. But to do system upgrade, it will also need space to download metadata from repositories and space needed by rpm to do the actual work - install and upgrade packages.

If you are struggling with free space on the root filesystem, there is newly available option --downloaddir usable to redirect the directory where packages are downloaded into (if you are interested, this is the pull request https://github.com/rpm-software-management/dnf-plugins-extras/pull/118 - not released in Fedora yet, available only in upstream). By default, packages are downloaded into /var/lib/dnf/system-upgrade.

thanks for the heads up. I'll keep an eye for when that makes it mainline. 

I will probably symlink /dnf/system-upgrade to somewhere will more free space. 

thanks.

I am updating to Fed28, so I used this as a chance to get another look at the problem:

[code]
Install     124 Packages
Upgrade    3230 Packages
Remove        3 Packages
Downgrade     8 Packages

Total download size: 4.1 G
DNF will only download packages, install gpg keys, and check the transaction.
[/code]

At this point I had 7.8GB free on root fs. 

[code]
Running transaction check
Transaction check succeeded.
Running transaction test
Error: Transaction check error:
  installing package kicad-1:4.0.7-3.fc28.x86_64 needs 1978MB on the / filesystem

Error Summary
-------------
Disk Requirements:
   At least 1978MB more space needed on the / filesystem.

[/code]

Problems: 
1)
After downloads , there is still 3.3 GB remaining. 7.8 - 4.1 = 3.7 , initial size requirement seems wrong. 

2) kicad does not need 2GB, this is false reporting of some other error, presumably a total requirement for the update. This needs to be correctly trapped and correctly reported as to what the REAL problem is. 

It is fairly simple to create a situation on a real installation or a VM test where there is insufficient disk space. This appears to have been coded without ever being tested.

du -h /var/lib/dnf/system-upgrade

4.3GB, near enough though a little more than predicted. The unpacking space needs to be assessed and correctly trapped.

In /var/lib/dnf/system-upgrade are two datasets:

1. downloaded packages (announced 4.1G)
2. repositories metadata cache (having only fedora and updates repositories enabled, the metadata consume 150M)

I'll look into rpm transaction check reporting another 2G. I suppose, the required space is calculated for the whole transaction (~3300 packages), not only for kicad package.

As the required disk space is computed by rpm, I'd like to ask what is the exact meaning of 'kicad-1:4.0.7-3.fc28.x86_64 needs 1978MB'. Are these 2G really required by kicad package, or does it represent a disk requirements for the whole transaction?

I suspect that error printed is triggered by a global calculation but is printing a package related one. 

I'm surprised about kicad wanting 2 GB, see top post where previous 26-27 upgrade it wanted 700MB , seems a heavy kicker , maybe they added another rake of library data. ( it's a PCB layout tool , could have thousands of component details. ) 

Also see the other package "needing" 33MB but but failed in a similar way. At that point there were 4GB available and it could not find 33 MB ?? 

IMO it is poor error trapping and reporting. There is a global lack of space which is detected but the error is being handled at the package level and falsely reports the pkg disk requirement.

Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

as reported above this also is seen upgrading 27 to 28. 

re-opening.

What the "needs XX MB on the / filesystem" rpm message means is that it needs XX MB more than is available. Which is exactly what *dnf* says on the matter:

Disk Requirements:
   At least 691MB more space needed on the / filesystem.

...which is inherited from yum, which started "translating" the raw rpm message at some point in history (it's somewhere in bugzilla) because of the misunderstandings. At which point it became harder to change the rpm message because yum was relying on the exact format... etc, convoluted history as is typical for anything around rpm :)

The problems are reported by package because ... that's how it's always been. And sure it could be smarter - sometimes you'd want the global summary, sometimes you'd want to know which package(s) to drop from the transaction to make it fit. Sometimes these even point out bugs in packaging (and rpm), 

And yes kicad is *big*, the package file alone is ~3GB, never mind expanded size, so I dont see any reason to doubt the reported sizes. Also note that the disk requirement during upgrade is actually roughly double the final requirement because of intermediate files.

thanks for some detail on that. 

From the initial report:

Error: Transaction check error:
  installing package kicad-1:4.0.7-1.fc27.x86_64 needs 691MB on the / filesystem
  installing package pocl-0.15-0.1.20171023git53ef5e8.fc27.x86_64 needs 33MB on the / filesystem


So if pocl says it needs 33MB more there is basically ZERO free space, if that error is accurate. Also from top post:

It appears to think the disk if entirely full which is incorrect: df shows about 4GB remaining. 

That makes no sense and is contradictory. Maybe dnf-system-upgrade is creating a temporary fs , or blocking disk space with a huge temp file so simulate expected usage , which it is deleting before returning due to the error. 

If that is the case some better disk space checking and error trapping is needed. I found it needed almost 2GB more before it would run OK. It should be possible to get the pre-installation checks to be more accurate than that. 

Taking to other extreme, kicad is almost certainly the largest individual pkg in this upgrade and wanted circa 700MB. Yet the real need was for 2GB more. An individual installation of kicad which reqd 700GB would run if I gave it 700GB. In this case freeing up progressively more space did not allow it go ahead until I had 5.9GB free.

As I said originally, the error being reported is clearly NOT the real error condition, nor is it reporting the real disk space required. Hence the title false accounting. 

Like I said, this clearly has not been stress tested on a real system to see what happens if this error condition is present.

Like I said in the previous comment (and what the dnf message tells you), the rpm message says you need that much *further* space compared to what you have now. 

The original message basically tells you:
- either free up >= 691MB on / or remove kicad prior to update
- either free up >= 33MB on / or remove pocl prior to update

Estimating disk space requirements is not an exact science because there's unknown amount of unaccounted space consumed as side-effect running scriptlets, writing logs, database growth (rpm tries to account for its own db growth) and other activity besides the transaction etc.

And again like said, I'm sure it all could be somehow presented better, but I don't see any obvious problem with the numbers here, just a misunderstanding on what they mean.

As there hasn't been any new evidence, closing as NOTABUG.

Also changing the actual message is not so simple because other software (at least yum) actually depends on the exact format.