Description of problem: When building image with Dockerfile that starts with FROM fedora:23 RUN dnf upgrade -y there are numerous warnings like /usr/share/locale/de/LC_MESSAGES/elfutils.mo: No such file or directory cannot reconstruct rpm from disk files /usr/share/man/man5/libaudit.conf.5.gz: No such file or directory cannot reconstruct rpm from disk files /usr/share/man/man8/ca-legacy.8.gz: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/coreutils/ABOUT-NLS: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/expat/README: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/libarchive/NEWS: No such file or directory cannot reconstruct rpm from disk files Version-Release number of selected component (if applicable): fedora:23 How reproducible: Deterministic. Steps to Reproduce: 1. Have Dockerfile FROM fedora:23 RUN dnf upgrade -y 2. Run docker build -t testdelta . Actual results: Total download size: 41 M Downloading Packages: /usr/share/locale/de/LC_MESSAGES/elfutils.mo: No such file or directory cannot reconstruct rpm from disk files /usr/share/man/man5/libaudit.conf.5.gz: No such file or directory cannot reconstruct rpm from disk files /usr/share/man/man8/ca-legacy.8.gz: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/coreutils/ABOUT-NLS: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/expat/README: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/libarchive/NEWS: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/libksba/AUTHORS: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/libtasn1/AUTHORS: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/libseccomp/CHANGELOG: No such file or directory cannot reconstruct rpm from disk files /usr/share/locale/fi/LC_MESSAGES/rpm.mo: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/shadow-utils/HOWTO: No such file or directory cannot reconstruct rpm from disk files /usr/share/doc/sqlite-libs/README.md: No such file or directory cannot reconstruct rpm from disk files /usr/lib64/python3.4/email/architecture.rst: No such file or directory cannot reconstruct rpm from disk files libtasn1-4.8-1.fc23.x86_64: Delta RPM rebuild failed libseccomp-2.3.1-0.fc23.x86_64: Delta RPM rebuild failed sqlite-libs-3.11.0-3.fc23.x86_64: Delta RPM rebuild failed elfutils-libelf-0.166-1.fc23.x86_64: Delta RPM rebuild failed ca-certificates-2016.2.7-1.0.fc23.noarch: Delta RPM rebuild failed shadow-utils-2:4.2.1-8.fc23.x86_64: Delta RPM rebuild failed coreutils-8.24-7.fc23.x86_64: Delta RPM rebuild failed python3-libs-3.4.3-6.fc23.x86_64: Delta RPM rebuild failed expat-2.1.1-1.fc23.x86_64: Delta RPM rebuild failed audit-libs-2.5.1-1.fc23.x86_64: Delta RPM rebuild failed rpm-4.13.0-0.rc1.13.fc23.x86_64: Delta RPM rebuild failed libksba-1.3.4-1.fc23.x86_64: Delta RPM rebuild failed libarchive-3.1.2-15.fc23.x86_64: Delta RPM rebuild failed -------------------------------------------------------------------------------- Total 2.5 MB/s | 41 MB 00:16 Delta RPMs reduced 40.7 MB of updates to 41.4 MB (-1.1% saved) Expected results: No delta compited. Additional info: With the default setup, more data is downloaded by dnf in an attempt to use drpms than if it was disabled, wasting bandwidth and CPU. Using --setopt=deltarpm=false works but it really should be default. This is a copy of https://fedorahosted.org/spin-kickstarts/ticket/56.
Sounds like this would be fixed by adding 'deltarpm=0' in the same place other dnf.conf changes are made in the kickstart. Would you like to submit a PR with that change ( https://pagure.io/fedora-kickstarts )? Or if you prefer I can whip up one...
I assume you mean around https://pagure.io/fedora-kickstarts/blob/master/f/fedora-docker-base.ks#_62 What is the example image which should carry these changes? Because $ docker run --rm -ti fedora:23 cat /etc/dnf/dnf.conf [main] gpgcheck=1 installonly_limit=3 clean_requirements_on_remove=True $ docker run --rm -ti fedora:rawhide cat /etc/dnf/dnf.conf [main] gpgcheck=1 installonly_limit=3 clean_requirements_on_remove=True $ Looking at the awk command, it's clear why it fails -- there is no empty line in that file (that the awk looks for with that NF==0). The yum.conf file that the .ks says attempts to make consistent with dnf.conf does not even exist: $ docker run --rm -ti fedora:rawhide cat /etc/yum.conf cat: /etc/yum.conf: No such file or directory
Yeah, I would drop the yum section, no idea why thats still around. I have no idea how long it's been broken, but it seems to me the awk should get fixed. Or perhaps replaced entirely with something less fragile? Perhaps just catting the contents desired into the file?
My main concern with that yum.conf is -- why didn't the build process fail with loud errors? Do we need some set -e there? As for the dnf.conf, yes, merely appending will likely work.
Not sure. Perhaps it's a oz bug? https://kojipkgs.fedoraproject.org//work/tasks/4960/14404960/oz-x86_64.log
Should I file pull request adding set -e to the %post, so that we start learning about all operations that we think pass but they don't? That's what I find nice about Dockerfile -- failed RUN fails the whole docker build, and you can write it very defensively, especially when adding workarounds or tweaks that might not be needed eventually.
Sounds like a good idea to me.
Filed https://pagure.io/fedora-kickstarts/pull-request/32.
Merged
Thanks. What is the best way to try the new .ks? Is there some koji / fedpkg method to run a scratch build?
It gets automatically used in the next rawhide compose... so, going to: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20160609.n.0/logs/global/pungi.global.log and looking for base docker we see: 2016-06-09 08:11:37 [DEBUG ] build-image outputs: {'output': 'Created task: 14433136\nTask info: http://koji.fedoraproject.org/koji/taskinfo?taskID=14433136 and looking at that tasks oz log: https://kojipkgs.fedoraproject.org//work/tasks/3165/14433165/oz-x86_64.log I do see the set -e in there. :)
Set -e is there but did it cause the build to fail? There are some FAILs in https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20160609.n.0/logs/global/pungi.global.log but the http://koji.fedoraproject.org/koji/taskinfo?taskID=14433136 is marked successful. We really wanted to see the task fail hard, didn't we?
It doesn't seem like it did cause it to fail... perhaps there's something about the oz env thats different/special. imcleod would be the one to check with there.
Ian, could you help us figure out how to harden the %post for docker images? We've added set -e but it did not seem to help. Thank you.
This message is a reminder that Fedora 23 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 23. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '23'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 23 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Issue still present on fedora:25 image.
Confirming that it happens to me too.
This message is a reminder that Fedora 25 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 25. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '25'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 25 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
The issue still seems to be present with fedora:26 images: $ docker build -t bz-1343114-26 . Sending build context to Docker daemon 2.048 kB Step 1/2 : FROM fedora:26 ---> 1ba39c7494fb Step 2/2 : RUN dnf upgrade -y ---> Running in 0829d8fa344f Fedora 26 - x86_64 - Updates 2.5 MB/s | 18 MB 00:07 Fedora 26 - x86_64 2.8 MB/s | 53 MB 00:18 Last metadata expiration check: 0:00:08 ago on Wed Nov 29 14:26:25 2017. [...] (1/18): openssl-libs-1.1.0f-7.fc26_1.1.0g-1.fc2 724 kB/s | 271 kB 00:00 (2/18): gawk-4.1.4-5.fc26_4.1.4-6.fc26.x86_64.d 858 kB/s | 335 kB 00:00 /usr/share/locale/vi/LC_MESSAGES/gawk.mo: No such file or directory cannot reconstruct rpm from disk files (3/18): gnupg2-2.2.1-1.fc26_2.2.3-1.fc26.x86_64 1.2 MB/s | 1.5 MB 00:01 /usr/share/doc/gnupg2/AUTHORS: No such file or directory cannot reconstruct rpm from disk files (4/18): coreutils-8.27-7.fc26.x86_64.rpm 974 kB/s | 1.1 MB 00:01 [...] -------------------------------------------------------------------------------- Total 2.4 MB/s | 22 MB 00:09 Failed Delta RPMs increased 21.3 MB of updates to 22.2 MB (-4.1% wasted) Running transaction check [...]
This message is a reminder that Fedora 26 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '26'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 26 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
The issue seems to be still present with fedora:27 image: $ docker run --rm -ti fedora:27 dnf upgrade -y [...] /usr/share/man/ca/man8/pam_sss.8.gz: No such file or directory ] 3.5 MB/s | 222 kB 00:02 ETA cannot reconstruct rpm from disk files Also, changing component in an attempt to find some place where issue with base container images are tracked.
This message is a reminder that Fedora 27 is nearing its end of life. On 2018-Nov-30 Fedora will stop maintaining and issuing updates for Fedora 27. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '27'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 27 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 27 changed to end-of-life (EOL) status on 2018-11-30. Fedora 27 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.