Description of problem: Last night's dnf upgrade gave error: Running scriptlet: glibc-2.25-7.fc26.x86_64 34/34BDB1539 Build signature doesn't match environment failed loading RPMDB Version-Release number of selected component (if applicable): glibc-2.25-8.fc26.x86_64 dnf-2.6.3-1.fc26.noarch How reproducible: Unknown Steps to Reproduce: 1. Do dnf upgrade Actual results: glibc script error. Expected results: No glibc script error. Additional info:
The error message comes from libdb, reassigning.
Thanks for reporting this. Looks like fallout from https://bugzilla.redhat.com/show_bug.cgi?id=1394862 Something seems to be trying to access the rpmdb after the rpm transaction goes through (before dnf's verify step) and finds itself unable to do so as glibc got updated. I am unable to reproduce this on a clean F26 box so it is possibly something specific to your box that is causing issues. Your best bet is to find that 'something' and stop it from accessing the rpmdb when it is not safe to do.
Created attachment 1316338 [details] Journal listing at time of problem As per the journal, something unusual happened during the middle of that upgrade.
After today's large update, including kernel 4.12.8-300.fc26.x86_64, I again received the same error: BDB1539 Build signature doesn't match environment failed loading RPMDB However, as far as I can tell, the upgrade was successful. Looking at the journal log, I didn't notice anything unusual.
Just now I got: ... Running scriptlet: kernel-core-4.11.11-300.fc26.x86_64 114/114 BDB1539 Build signature doesn't match environment failed loading RPMDB The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'.
This can very easily be reproduced in F27 container. Just do: 1. docker run --rm -ti fedora:27 bash 2. dnf upgrade -y glibc
I'm also seeing this on both f26 and f27.
(In reply to Stanislav Laznicka from comment #6) >This can very easily be reproduced in F27 container. Thanks Standa for providing the reproducer. This can even be reproduced exactly the same way with f26 version of the image... Will look into this.
This is also affecting our CI infra which runs builds in Docker containers. To clarify, it looks like the scriptlet of 2.25-9 is causing the failure, which is present in the latest fedora:26 image.
Looks like a container specific issue. The reason this is happening is because one of the hacks introduced via the fix for bug 1394862 depends on being able to get pid of the process that has rpmdb's environment locked for reading, which is done by parsing /proc/locks looking for the inode of the file of interest. Unfortunately, for some reason (possibly a feature of OverlayFS?), in docker containers the inode of the file that is actually locked does not correspond to its entry in /proc/locks: $ touch test; inode=`ls test -i | awk '{print $1}'`; flock test bash -c "cat /proc/locks" | grep $inode prints nothing inside a docker container, while working correctly outside of it.
(In reply to Petr Kubat from comment #10) > Looks like a container specific issue. Occurred at the end of a DNF update on a normal (non-container) install for me.
>Occurred at the end of a DNF update on a normal (non-container) install for me. Not saying this might not occur on a non-container system, but the causes for it would likely be different. I have so far been unable to reproduce this outside containers on a clean system with only a glibc update so if you are running into the same failure on your system it might be something specific to your configuration that is the culprit. FYI: Not using overlay2 as the storage driver for your docker containers can be used as a workaround for the time being.
I also had this happen on with a f26 docker container and on f27 with a regular upgrade. Changing the docker storage driver is not an option right now.
At least for the docker case, this should presumably be fixed once a new base container with the glibc update comes out? Or is it related to updating glibc *at all*, not just this particular glibc update?
Also reproduced with a f25 container. I guess since the fix in bug 1394862 went into f25 as well.
(In reply to Colin Walters from comment #14) >Or is it related to updating glibc *at all*, not just this particular glibc update? Exactly. Since we do not know if a glibc update might change some internal structures libdb is using (and introduce another bug 1394862), libdb tries to rebuild its environment every time a new glibc rpm is installed and fails if it finds itself not being able to do this safely. (In reply to Peter from comment #15) > Also reproduced with a f25 container. I guess since the fix in bug 1394862 > went into f25 as well. This issue is a combination of three things - one is the libdb bug you have linked, which tries to look on /proc/locks, and went into all supported Fedora version at that time, to make sure the upgrade process goes through. Second is the recent F26 change to make overlay2 the default storage engine for docker and the last is overlay itself, since the inodes it uses for locking (and populating /proc/locks) do not correspond to the real inodes of the files being locked, which as I understand it now might be a feature of overlay (I have asked the maintainer for comment).
I think the best way of moving this forward is to assign the bug to dnf as it is in dnf where a fix makes the most sense. To sum up for dnf maintainers what this bug is about: The upstream fix for bug 1394862 (back-ported to our version of libdb) made libdb able to automatically rebuild its environment when trying to attach to it in case the environment was created by an older version of libdb, based on internal structure signature, version numbers, or pthread modify timestamp change. Of course the rebuild would only be safe when there is no other process accessing the environment, which is why libdb has been modified to also try to lock its first region (__db.001) via a flock-style lock (since fcntl-style locks get released on close(2)). Unfortunately, since dnf still accesses the rpmdb in its verification step, after the transaction goes through successfully, and libdb only looks at the modify timestamp (which has by that time changed due to the update), a specific hack was needed to make sure dnf does not fail in its verification (post-transaction) phase because of libdb thinking it is trying to access an old environment based on the timestamp (which is not true, as the process should still have the old pthreads library loaded at that point). The hack itself tries to look-up the inode of the first region file in `/proc/locks` to get the PID of the process holding the lock (since that information cannot be obtained any other way for flock-style locks afaik) and compares it with the PID of the running process to make sure it is still safe to access the environment. Ugly as this hack is, it has worked until now when it was found that things still break when trying to do an update in containerized fedora as in Fedora 26 there was a change of the default docker storage driver from devicemapper to overlay, which has the unfortunate feature of not having the inodes of locked files found in `/proc/locks` correspond to the inodes of real files. It is my understanding that this is by design, so that file locks are preserved even during copy-up of files from lower layers. The combination of these things makes the hack fail to make sure the process is safe to access the environment and bail immediately. Additionally, there have been reports of similar failures even in non-containerized Fedora system (comment 11, comment 13) but I have not been able to reproduce these failures so far. Fixing the issue in dnf would, however, most likely result in getting rid of these failures as well.
Adding a "Regression" keyword as building containers that contain "dnf upgrade -y" was possible previously but is not anymore.
This error appears in Fedora 25 containers as well.
(In reply to Stanislav Laznicka from comment #6) > This can very easily be reproduced in F27 container. > > Just do: > 1. docker run --rm -ti fedora:27 bash > 2. dnf upgrade -y glibc Do you have latest images? (In reply to Stanislav Laznicka from comment #18) > Adding a "Regression" keyword as building containers that contain "dnf > upgrade -y" was possible previously but is not anymore. I tested on f27 and following image works for me registry.fedoraproject.org/fedora 27 1e635fb211cd I do not want to say it is not a bug. But there might be also another missing part for triggering it. And glic was updates :-) in my case
(In reply to Lukas Slebodnik from comment #20) > I tested on f27 and following image works for me > registry.fedoraproject.org/fedora 27 1e635fb211cd > > I do not want to say it is not a bug. But there might be also another > missing part for triggering it. And glic was updates :-) in my case I have been able to reproduce this on every clean-installed post f25 system I have tried, including rawhide. Is your docker daemon using the default overlay as its storage driver? Can check with: $ docker info | grep Storage --- Panu: I have also checked that updating to newest rpm release first does not help.
(In reply to Lukas Slebodnik from comment #20) > (In reply to Stanislav Laznicka from comment #6) > > This can very easily be reproduced in F27 container. > > > > Just do: > > 1. docker run --rm -ti fedora:27 bash > > 2. dnf upgrade -y glibc > > Do you have latest images? > Yes. I have the same image you do. # docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.fedoraproject.org/fedora 27 1e635fb211cd 13 days ago 233 MB > (In reply to Stanislav Laznicka from comment #18) > > Adding a "Regression" keyword as building containers that contain "dnf > > upgrade -y" was possible previously but is not anymore. > > I tested on f27 and following image works for me > registry.fedoraproject.org/fedora 27 1e635fb211cd I don't know about f27, this happens on F26 for me. If it works in F27, it might be worth checking out for anyone fixing this. > > I do not want to say it is not a bug. But there might be also another > missing part for triggering it. And glic was updates :-) in my case I think we can safely say it's a bug. I am not the only person who reproduced it should you skim through this BZ. It's also reproducible 100% of the time on F26.
(In reply to Petr Kubat from comment #21) > (In reply to Lukas Slebodnik from comment #20) > > I tested on f27 and following image works for me > > registry.fedoraproject.org/fedora 27 1e635fb211cd > > > > I do not want to say it is not a bug. But there might be also another > > missing part for triggering it. And glic was updates :-) in my case > > I have been able to reproduce this on every clean-installed post f25 system > I have tried, including rawhide. > > Is your docker daemon using the default overlay as its storage driver? Can > check with: > $ docker info | grep Storage > That is the case. I tested on two different machines and both of them have different storage driver :-) [root@f27 ~]# docker info | grep Storage Storage Driver: btrfs [root@f25 ~]# docker info | grep Storage Storage Driver: devicemapper
(In reply to Petr Kubat from comment #17) > Ugly as this hack is, it has worked until now when it was found that things > still break when trying to do an update in containerized fedora as in Fedora > 26 there was a change of the default docker storage driver from devicemapper > to overlay, which has the unfortunate feature of not having the inodes of > locked files found in `/proc/locks` correspond to the inodes of real files. > It is my understanding that this is by design, One thing that might help is to force a copy up the first time we open the rpmdb for write. Then we only lock the overlay file. This is effectively what rpm-ostree does: https://github.com/projectatomic/rpm-ostree/blob/79086f8c3d3037828e81372439f3a68c49e6d049/src/libpriv/rpmostree-core.c#L3086 (rpm-ostree uses ostree, not overlayfs, but the idea is the same) Something like this in pseudocode: lock_and_copyup() { lock(/var/lib/rpm) if is_overlayfs(/var/lib/rpm) && !exists(/var/lib/rpm/.copyup) { for file in /var/lib/rpm { force_copyup (file) } touch(/var/lib/rpm/.copyup) } } unlock(/var/lib/rpm) lock(/var/lib/rpm) } (Worth noting that host systems managed via rpm-ostree are pretty much systematically immune to issues like this one due to the way we do offline updates/installs by default, and only optionally make them live after everything is complete - that plus the fact that the rpm database always appears read-only to everything except rpm-ostree (and we always regenerate it fresh) and there are no plugins etc.)
(In reply to Colin Walters from comment #24) >One thing that might help is to force a copy up the first time we open the rpmdb for write. Then we only lock the overlay file. The file that is being locked by libdb is already present in the upper layer so I guess this will not work for overlay. --- The maintainer for the overlay part of the kernel already sent me a patch (I have yet to test) so I guess at least the containerized part of this issue can be thought of as a bug in the kernel.
(In reply to Petr Kubat from comment #21) > I have been able to reproduce this on every clean-installed post f25 system > I have tried, including rawhide. Maybe "clean-installed" has something to do with it (this laptop was a fresh install of Fedora 15, many moons ago), but I cannot reproduce the docker case at all. Which is really annoying since ... well, debugging without a reproducer is. > Is your docker daemon using the default overlay as its storage driver? Can > check with: > $ docker info | grep Storage [root@sopuli ~]# docker info | grep Storage WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device. Storage Driver: devicemapper WARNING: You're not using the default seccomp profile Whatever that means. I'm pretty sure I haven't manually changed that but then this has been updated through eleven Fedora versions... Doing a fresh install of fedora 26 in libvirt gives something different (Storage Driver: overlay2), and there it is reproducable. Yay, progress - this means I will be able to actually look into this. Thanks for the tip, wouldn't have guessed that in a million years. BUT. That also sounds somehow awfully ominuous. Guess it's time to go read what these different storage options actually do and why does overlay2 screw up rpm so. Not that docker and rpm ever liked each-other too much. (after hitting mid-air collision and seeing the last update: possible kernel bug, so sounds ominuous indeed)
(In reply to Panu Matilainen from comment #26) > BUT. That also sounds somehow awfully ominuous. Guess it's time to go read > what these different storage options actually do and why does overlay2 screw > up rpm so. Not that docker and rpm ever liked each-other too much. I have already touched up on that a bit in comment 16 and comment 17. My understanding is that overlayfs stores the overlay inodes of locked files into `/proc/locks` instead of inodes of real files located in upper/lower layers of the filesystem. I do not think this is documented anywhere apart from the kernel patch that introduced this (and I do hope this is the one): https://patchwork.kernel.org/patch/9236917/
(In reply to Petr Kubat from comment #27) > I have already touched up on that a bit in comment 16 and comment 17. Right, so you have, and pointed out the exact workaround in comment 12. My bad for missing those. > My understanding is that overlayfs stores the overlay inodes of locked files > into `/proc/locks` instead of inodes of real files located in upper/lower > layers of the filesystem. I do not think this is documented anywhere apart > from the kernel patch that introduced this (and I do hope this is the one): > > https://patchwork.kernel.org/patch/9236917/ Right. In the meanwhile, now that I have a reproducer... The source of this particular error is actually libsolv, which uses its own code for reading the rpmdb. Which explains why the band-aid in rpm wasn't working for this case.
Created attachment 1323628 [details] libsolv patch to fall back to DB_PRIVATE on DB_VERSION_MISMATCH The attached patch does basically the same as rpm in this situation: it falls back to using DB_PRIVATE environment. It's not really a safe thing to do so it's band-aid rather than actual solution, but it lets the operation complete: Running transaction Preparing : 1/1 Upgrading : glibc-common-2.26-7.fc27.x86_64 1/8 Upgrading : glibc-langpack-en-2.26-7.fc27.x86_64 2/8 Running scriptlet: glibc-2.26-7.fc27.x86_64 3/8 Upgrading : glibc-2.26-7.fc27.x86_64 3/8 Running scriptlet: glibc-2.26-7.fc27.x86_64 3/8 Upgrading : libcrypt-nss-2.26-7.fc27.x86_64 4/8 Running scriptlet: libcrypt-nss-2.26-7.fc27.x86_64 4/8 Cleanup : libcrypt-nss-2.26-4.fc27.x86_64 5/8 Running scriptlet: libcrypt-nss-2.26-4.fc27.x86_64 5/8 Cleanup : glibc-2.26-4.fc27.x86_64 6/8 Running scriptlet: glibc-2.26-4.fc27.x86_64 6/8 Cleanup : glibc-langpack-en-2.26-4.fc27.x86_64 7/8 Cleanup : glibc-common-2.26-4.fc27.x86_64 8/8 BDB1539 Build signature doesn't match environment Verifying : glibc-2.26-7.fc27.x86_64 1/8 Verifying : glibc-common-2.26-7.fc27.x86_64 2/8 Verifying : glibc-langpack-en-2.26-7.fc27.x86_64 3/8 Verifying : libcrypt-nss-2.26-7.fc27.x86_64 4/8 Verifying : glibc-common-2.26-4.fc27.x86_64 5/8 Verifying : glibc-langpack-en-2.26-4.fc27.x86_64 6/8 Verifying : libcrypt-nss-2.26-4.fc27.x86_64 7/8 Verifying : glibc-2.26-4.fc27.x86_64 8/8 Upgraded: glibc.x86_64 2.26-7.fc27 glibc-common.x86_64 2.26-7.fc27 glibc-langpack-en.x86_64 2.26-7.fc27 libcrypt-nss.x86_64 2.26-7.fc27
I can confirm the patch works in in my containers, thanks! Hopefully this will also fix the non-containerized use-case, but I have no way to check as I lack a reproducer. For the people that were affected - are you able to test this? Anyway I think libsolv/dnf should also log some info message that it is trying to fall back to DB_PRIVATE. Having an error message right at the start of the verification step might scare the people that pay attention to its output.
A scratch-build of libsolv with the above patch: https://koji.fedoraproject.org/koji/taskinfo?taskID=21724101 And yeah it needs a diagnostic message, both in rpm itself and libsolv (dnf doesn't have the slightest clue of what's going on here). The problem is how on earth to explain the mess in a oner-liner message... Looking past immediate workarounds: Dnf level is blissfully ignorant of all this, it just wants to open the system repo (aka rpmdb) to do the verify walk. It could also use native rpm iterators instead of libsolv to avoid this to begin with. Anyway, when invoked at this point, libsolv rpmdb extension notices the rpmdb has changed since the solv file creation and wants to recreate it. And for that it opens up a new db environment to do it because it has no other choice, it's not aware of the upper layers and doesn't even use librpm code for it. If dnf closed ALL database references before doing this, it should actually be ok, but this is skating on thin ice no matter what. One possibility would be turning the picture upside down by implementing the libsolv solve-file creation as an rpm plugin which could use the already existing environment within rpm, but that'd be far more work of course.
>If dnf closed ALL database references before doing this, it should actually be ok, but this is skating on thin ice no matter what. If dnf closed all its references on rpm's environment nothing would obstruct libsolv to rebuild the environment during a glibc upgrade, still with the old pthread library loaded in, and update the pthread timestamp to the newer one's, effectively re-introducing bug 1394862, since the timestamp is the only way we know we might not have a compatible environment already.
Ugh, indeed. Forgot about libsolv still using the old library in this case - the ice is even thinner than I remembered... So actually dnf would need to run the verification step as new, separately exec'ed process after closing all db references for it to work. Seems cumbersome but OTOH it'd make the rpmdb accessible earlier for other players too. Another radical idea is to throw out the whole verification step, I never liked it in yum either and AFAIK dnf simply inherited it from there.
(In reply to Panu Matilainen from comment #33) > So actually dnf would need to run the verification step as new, separately > exec'ed process after closing all db references for it to work. Seems > cumbersome but OTOH it'd make the rpmdb accessible earlier for other players > too. Making the rpmdb accessible just before the verification step might make the verification fail in case some other process gets write access to the rpmdb though. Well depending on what is checked during the verification step I guess. Making libsolv able to use dnf's environment handle would be the best way to fix this but I can see how it would be a lot more work than any other solution so far.
I installed F27 fresh on my laptop and trying to create my own docker images again on this system but the images which tries to install gcc or glibc are failing to build the image with the "BDB1539 Build signature doesn't match environment Failed loading RPMDB." error. If there is any test packages to test for fixing this issue then I will be happy to test it.
I was able to run "dnf update" (I built this image [1]) in F26AH using devicemapper. With overlay2 it is NOT working. [1] https://github.com/projectatomic/atomic-system-containers/blob/master/kubernetes-master/Dockerfile
*** Bug 1483854 has been marked as a duplicate of this bug. ***
I've refreshed all Fedora base images in both the Docker Hub[0] and in registry.fedoraproject.org, are these issues still persisting? [0] - https://github.com/docker-library/official-images/pull/3459
It looks like Spyros broke this case (good detective work!). I'm leaning more towards the non POSIX compliant parts of overlayfs now. To be clear, this is a problem when using overlayfs as the docker storage backend and IS NOT a problem when using devicemapper. This single command on F26AH with overlay2 yields: ``` # docker run -it --rm registry.fedoraproject.org/fedora:rawhide dnf update -y ... Running scriptlet: glibc-common-2.26.90-14.fc28.x86_64 112/112 Failed to connect to bus: No such file or directory BDB1539 Build signature doesn't match environment failed loading RPMDB The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'. ``` With devicemapper as the backend and that same command we get a successful update.
(In reply to Spyros Trigazis from comment #36) > I was able to run "dnf update" (I built this image [1]) in F26AH using > devicemapper. With overlay2 it is NOT working. > > [1] > https://github.com/projectatomic/atomic-system-containers/blob/master/ > kubernetes-master/Dockerfile Indeed, as stated in comment 12. The person maintaining ovrerlayfs upstream already created a patch for the issue. Not sure how quickly it will get integrated though... (In reply to Adam Miller from comment #38) > I've refreshed all Fedora base images in both the Docker Hub[0] and in > registry.fedoraproject.org, are these issues still persisting? > > [0] - https://github.com/docker-library/official-images/pull/3459 Not persisting as in there is no glibc update right now that can break things via "dnf update" but running "dnf downgrade glibc" still runs into the same issues. I guess the libsolv band-aid is not yet in?
IIUC, it is similar issue we faced in the past. That is a file on lower filesystem is opened for READ. And later another process opens it for WRITE and file get copied up. But now reader and writer are looking at two different copies and any updates by writer are not visible to reader. This is an issue with overlay filesystem (as of now) and people found workarounds for it. Some people modified their docker file to touch rpm files. RUN touch /var/lib/rpm/* IIRC, people also wrote a yum/dnf plugin which touch rpm files on startup. That makes sure files get copied up before any reader opens them and avoid this situation. Jeremy Eder will know more details. He was very involved with that work.
Jeremy, IIRC, you and yum/dnf team already solved this issue. Did all the work you guys did, make in fedora as well?
The original BZ is https://bugzilla.redhat.com/show_bug.cgi?id=1213602 The solution was a yum ovl plugin, the mechanics of which are essentially what Colin has mentioned in c#24.
And the Fedora equivalent was https://bugzilla.redhat.com/show_bug.cgi?id=1216064
one more interesting tidbit: I tried to `touch /var/lib/rpm/*` before running `dnf update -y` and I still get the `BDB1539 Build signature doesn't match environment` error. So 1 docker run -it --rm registry.fedoraproject.org/fedora:rawhide 2 touch /var/lib/rpm/* 3 dnf update -y Still see problem.
(In reply to Dusty Mabe from comment #45) > one more interesting tidbit: > > I tried to `touch /var/lib/rpm/*` before running `dnf update -y` and I still > get the `BDB1539 Build signature doesn't match environment` error. > > So > > 1 docker run -it --rm registry.fedoraproject.org/fedora:rawhide > 2 touch /var/lib/rpm/* > 3 dnf update -y > > Still see problem. That's because it is a different issue... See comment 17 for the reason the error is coming up.
I am sorry, but I cannot reproduce the issue according to Comment 6, Comment 8, Comment 39, and Comment 45. Comment 10 also describe that the issue is caused by scriplet and performing scriplet is not job of dnf, but rpm. But in such an issue even RPM cannot solve the issue but change in scriplet definitely help. To Comment 17: I still don't see any problem in dnf, mostly due I am unable to reproduce the issue with the present environment. If I will have a reproducer with failing dnf, I can do more for you. What I can do, I can refactor verification in dnf. Anyway now we depend on errors reported by RPM, and verification by comparison of requested rpm changes with new status of rpm-db is redundant (but in past it discovered some issues). At the present time we do not cache system repo to prevent problem with detection of minor changes in rpm-db. Unfortunately sum plugins also ask for its copy of installed rpm, therefore I cannot promise that there will be no other opener.
>If I will have a reproducer with failing dnf, I can do more for you. Step 1: Make sure you are using overlay2 (or overlay) as your storage driver for Docker (default in F26) $ docker info | grep Storage Storage Driver: overlay Step 2: As there is no glibc update available right now you have to trigger the issue by downgrading. $ docker run --rm -i fedora:26 dnf -y downgrade glibc Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Downgrading : glibc-common-2.25-6.fc26.x86_64 1/8 Downgrading : glibc-langpack-en-2.25-6.fc26.x86_64 2/8 Running scriptlet: glibc-2.25-6.fc26.x86_64 3/8 Downgrading : glibc-2.25-6.fc26.x86_64 3/8 Running scriptlet: glibc-2.25-6.fc26.x86_64 3/8 Downgrading : libcrypt-nss-2.25-6.fc26.x86_64 4/8 Running scriptlet: libcrypt-nss-2.25-6.fc26.x86_64 4/8 Erasing : libcrypt-nss-2.25-10.fc26.x86_64 5/8 Running scriptlet: libcrypt-nss-2.25-10.fc26.x86_64 5/8 Erasing : glibc-common-2.25-10.fc26.x86_64 6/8 Erasing : glibc-langpack-en-2.25-10.fc26.x86_64 7/8 Erasing : glibc-2.25-10.fc26.x86_64 8/8 Running scriptlet: glibc-2.25-10.fc26.x86_64 8/8 BDB1539 Build signature doesn't match environment failed loading RPMDB The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'. --- This reproducer should work for you even with the current state of the environment. However, if you still have trouble reproducing, let me know and I will try to help out.
(In reply to Jaroslav Mracek from comment #47) > I am sorry, but I cannot reproduce the issue according to Comment 6, Comment > 8, Comment 39, and Comment 45. Comment 10 also describe that the issue is > caused by scriplet and performing scriplet is not job of dnf, but rpm. But > in such an issue even RPM cannot solve the issue but change in scriplet > definitely help. The reason why you can't reproduce it is in Comment 38. Just do what Petr Kubat tells you in Comment 48.
(In reply to Jaroslav Mracek from comment #47) > What I can do, I can refactor verification in dnf. Anyway now we depend on > errors reported by RPM, and verification by comparison of requested rpm > changes with new status of rpm-db is redundant (but in past it discovered > some issues). At the present time we do not cache system repo to prevent > problem with detection of minor changes in rpm-db. Unfortunately sum plugins > also ask for its copy of installed rpm, therefore I cannot promise that > there will be no other opener. Don't do anything hasty, this is a complex and terribly subtle mess and it's quite easy to make things actually worse (see eg comments from 29 to 34)
libsolv-0.6.29-2.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-1c33746e7e
libsolv-0.6.29-2.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-bd488c853f
libsolv-0.6.29-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-e3e612f6a1
We need to make sure this fix gets into RHEL7.5 release. since we hope to move the default for container runtimes to OVERLAY for RHEL7.
(In reply to Daniel Walsh from comment #54) > We need to make sure this fix gets into RHEL7.5 release. since we hope to > move the default for container runtimes to OVERLAY for RHEL7. Um, RHEL-7 doesn't have glibc 2.25 so it's not affected by bug 1394862, so this particular thing is not happening there at all. Unless the glibc + libdb changes have been backported that is, and I'd be quite surprised if they were.
(In reply to Daniel Walsh from comment #54) > We need to make sure this fix gets into RHEL7.5 release. since we hope to > move the default for container runtimes to OVERLAY for RHEL7. I don't think that will pose any issues. The libdb hack mentioned in comment 17 (being the reason behind dnf's failuresin this BZ) is not present in rhel7 as of now. And I do hope it stays way.
Excellent thanks.
libsolv-0.6.29-2.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-1c33746e7e
libsolv-0.6.29-2.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-e3e612f6a1
libsolv-0.6.29-2.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-bd488c853f
libsolv-0.6.29-2.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
1. start docker container fedora:rawhide , dnf update, works 2. start docker container fedora:26 , upgrade to libsolv-0.6.29-2.fc26 , dnf update works 3. start docker container fedora:26 , upgrade to libsolv-0.6.29-2.fc26 , downgrade glibc works 4. start docker container fedora:26 , dnf update shows the error message "BDB1539 Build signature doesn't match environment" but running dnf update says "Nothing to do." with no errors.
kde CI using f26 docker image with (libsolv-0.6.29-2.fc26 updated) is still seeing errors as well, for example: https://build.kde.org/view/CI%20Management/job/Docker%20Generate%20FedoraQt5.8%20Image/54/console Re-opening
(In reply to Spyros Trigazis from comment #62) > 1. start docker container fedora:rawhide , dnf update, works > 2. start docker container fedora:26 , upgrade to libsolv-0.6.29-2.fc26 , dnf > update works > 3. start docker container fedora:26 , upgrade to libsolv-0.6.29-2.fc26 , > downgrade glibc works > 4. start docker container fedora:26 , dnf update shows the error message > "BDB1539 Build signature doesn't match environment" but running dnf update > says "Nothing to do." with no errors. 4. start docker container fedora:26 , dnf update shows the error message "BDB1539 Build signature doesn't match environment" but running dnf update *again* says "Nothing to do." with no errors. I guess it upgraded libsolv during the first run.
Folks, see comment #29: the updated libsolv doesn't make the error message go away, but it does let the update complete. And BTW ignatenko, this is exactly why the diagnostic message would've been important...
Yep, libdb still fails even with the libsolv update but the verification step completes since libsolv tries to continue using a private (non-shared) libdb environment. FYI: The overlayfs mantainer got back to me with the link to the kernel patch fixing the /proc/locks content: https://lkml.org/lkml/2017/9/20/244
libsolv-0.6.29-2.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
libsolv-0.6.29-2.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Re-opening (again) per comment #63
Hi Rex, I believe the problem was not that dnf/libsolv were not fixed, I think you just needed a rebased container image (which @maxamillion released yesterday) that had the newer builds. I.e., I think you were running the old broken ones instead of the fixed ones. I had this same problem on Bodhi's CI system, and I just confirmed that the new images seem to resolve it: https://github.com/fedora-infra/bodhi/pull/1915 You can see the build passed here: https://ci.centos.org/job/bodhi-bodhi/614/
Not sure if it's related, but I noticed that packagekitd had been consuming 100% CPU for a couple of days, since this particular issue occurred on one of my systems. Killing packagekitd resolved the problem, $ sudo dnf clean dbcache error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch error: cannot open Packages index using db5 - (-30969) error: cannot open Packages database in /var/lib/rpm Error: Error: rpmdb open failed $ sudo dnf update error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch error: cannot open Packages index using db5 - (-30969) error: cannot open Packages database in /var/lib/rpm Error: Error: rpmdb open failed $ sudo killall -KILL packagekitd $ sudo dnf clean dbcache 90 files removed $ dnf update ... FWIW, the packagekitd process was holding a lock in /proc/locks
The fixed image does not appear to be on dockerhub yet - I had to use FROM registry.fedoraproject.org/fedora:26 to get dnf working in images again.
For sure not available from dockerhub: https://gitlab.com/minishift/minishift/-/jobs/38406683 Any ETA? (or possibility to sync these automagically?)
+1 on needing this fixed for fedora:26, which is currently failing: $ docker run --rm -ti fedora:26 bash [root@469218cae304 /]# dnf upgrade -y glibc Fedora 26 - x86_64 - Updates 573 kB/s | 16 MB 00:28 Fedora 26 - x86_64 478 kB/s | 53 MB 01:54 Last metadata expiration check: 0:00:21 ago on Thu Nov 2 16:30:44 2017. Dependencies resolved. ==================================================================================================================================================== Package Arch Version Repository Size ==================================================================================================================================================== Upgrading: glibc x86_64 2.25-12.fc26 updates 3.4 M glibc-common x86_64 2.25-12.fc26 updates 890 k glibc-langpack-en x86_64 2.25-12.fc26 updates 291 k libcrypt-nss x86_64 2.25-12.fc26 updates 54 k Transaction Summary ==================================================================================================================================================== Upgrade 4 Packages Total download size: 4.6 M Downloading Packages: (1/4): glibc-langpack-en-2.25-12.fc26.x86_64.rpm 58 kB/s | 291 kB 00:05 (2/4): libcrypt-nss-2.25-12.fc26.x86_64.rpm 50 kB/s | 54 kB 00:01 (3/4): glibc-common-2.25-12.fc26.x86_64.rpm 131 kB/s | 890 kB 00:06 (4/4): glibc-2.25-12.fc26.x86_64.rpm 184 kB/s | 3.4 MB 00:19 ---------------------------------------------------------------------------------------------------------------------------------------------------- Total 240 kB/s | 4.6 MB 00:19 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Upgrading : glibc-common-2.25-12.fc26.x86_64 1/8 Upgrading : glibc-langpack-en-2.25-12.fc26.x86_64 2/8 Running scriptlet: glibc-2.25-12.fc26.x86_64 3/8 Upgrading : glibc-2.25-12.fc26.x86_64 3/8 Running scriptlet: glibc-2.25-12.fc26.x86_64 3/8 Upgrading : libcrypt-nss-2.25-12.fc26.x86_64 4/8 Running scriptlet: libcrypt-nss-2.25-12.fc26.x86_64 4/8 Cleanup : libcrypt-nss-2.25-10.fc26.x86_64 5/8 Running scriptlet: libcrypt-nss-2.25-10.fc26.x86_64 5/8 Cleanup : glibc-common-2.25-10.fc26.x86_64 6/8 Cleanup : glibc-langpack-en-2.25-10.fc26.x86_64 7/8 Cleanup : glibc-2.25-10.fc26.x86_64 8/8 Running scriptlet: glibc-2.25-10.fc26.x86_64 8/8 BDB1539 Build signature doesn't match environment failed loading RPMDB The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'.
(In reply to Mark Hirota from comment #74) > +1 on needing this fixed for fedora:26, which is currently failing: > > $ docker run --rm -ti fedora:26 bash [...] > BDB1539 Build signature doesn't match environment > failed loading RPMDB This is the symptoms when running with libsolv-0.6.29-1. With the above reproducer command I'm getting image sha256:64907a529db7e1b25d76c84928e2e1a82889c92a23f73b8f16385d05705ecea6 in which libsolv-0.6.29-2.fc26.x86_64 is present, and the upgrade succeeds despite the "build signature doesn't match" complaint: [...] Running scriptlet: glibc-2.25-10.fc26.x86_64 8/8 BDB1539 Build signature doesn't match environment Verifying : glibc-2.25-12.fc26.x86_64 1/8 Verifying : glibc-common-2.25-12.fc26.x86_64 2/8 Verifying : glibc-langpack-en-2.25-12.fc26.x86_64 3/8 Verifying : libcrypt-nss-2.25-12.fc26.x86_64 4/8 Verifying : glibc-common-2.25-10.fc26.x86_64 5/8 Verifying : glibc-langpack-en-2.25-10.fc26.x86_64 6/8 Verifying : libcrypt-nss-2.25-10.fc26.x86_64 7/8 Verifying : glibc-2.25-10.fc26.x86_64 8/8 [...] So before complaining about not working, everybody PLEASE make sure you have the updated libsolv version. Oh and there's a related issue on rpm-side too (there's an update pending stable), but hitting that would manifest in a different way.
(In reply to Panu Matilainen from comment #75) > So before complaining about not working, everybody PLEASE make sure you have > the updated libsolv version. Oh and there's a related issue on rpm-side too > (there's an update pending stable), but hitting that would manifest in a > different way. Hi, I'm not sure what you mean "make sure you have the updated libsolv version"? I tried both in the host environment (which happens to be F26) and inside the docker image build (Dockerfile) -- but it still fails. I am using the base docker fedora:26 image from hub.docker.com (Please see my config below). $ sudo dnf update -y libsolv Last metadata expiration check: 0:00:00 ago on Fri 03 Nov 2017 07:07:45 AM PDT. Dependencies resolved. ================================================================================ Package Arch Version Repository Size ================================================================================ Upgrading: libsolv x86_64 0.6.30-1.fc26 updates 365 k Transaction Summary ================================================================================ Upgrade 1 Package Total download size: 365 k Downloading Packages: libsolv-0.6.30-1.fc26.x86_64.rpm 459 kB/s | 365 kB 00:00 -------------------------------------------------------------------------------- Total 357 kB/s | 365 kB 00:01 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Upgrading : libsolv-0.6.30-1.fc26.x86_64 1/2 Running scriptlet: libsolv-0.6.30-1.fc26.x86_64 1/2 Cleanup : libsolv-0.6.28-5.fc26.x86_64 2/2 Running scriptlet: libsolv-0.6.28-5.fc26.x86_64 2/2 Verifying : libsolv-0.6.30-1.fc26.x86_64 1/2 Verifying : libsolv-0.6.28-5.fc26.x86_64 2/2 Upgraded: libsolv.x86_64 0.6.30-1.fc26 Complete! $ docker build --tag=disk-monitor . Sending build context to Docker daemon 5.12kB Step 1/9 : FROM fedora:26 ---> b0b140824a48 Step 2/9 : WORKDIR /mnt/work ---> Using cache ---> 8058d14d1ecb Step 3/9 : RUN dnf -y install binutils curl findutils gzip libsolv tar unzip wget which ---> Using cache ---> 05b2c3c22b1b Step 4/9 : RUN dnf -y install automake cmake gcc gcc-c++ glibc make ---> Running in 1d88bba9b423 Last metadata expiration check: 0:04:02 ago on Fri Nov 3 14:03:53 2017. Package glibc-2.25-10.fc26.x86_64 is already installed, skipping. Dependencies resolved. ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: automake noarch 1.15-9.fc26 fedora 695 k cmake x86_64 3.9.1-1.fc26 updates 7.4 M gcc x86_64 7.2.1-2.fc26 updates 21 M gcc-c++ x86_64 7.2.1-2.fc26 updates 11 M make x86_64 1:4.2.1-2.fc26 fedora 492 k Upgrading: glibc x86_64 2.25-12.fc26 updates 3.4 M glibc-common x86_64 2.25-12.fc26 updates 890 k glibc-langpack-en x86_64 2.25-12.fc26 updates 291 k libcrypt-nss x86_64 2.25-12.fc26 updates 54 k libgcc x86_64 7.2.1-2.fc26 updates 90 k Installing dependencies: autoconf noarch 2.69-24.fc26 fedora 709 k cmake-data noarch 3.9.1-1.fc26 updates 1.2 M cmake-filesystem x86_64 3.9.1-1.fc26 updates 35 k cpp x86_64 7.2.1-2.fc26 updates 9.2 M gc x86_64 7.6.0-2.fc26 fedora 108 k glibc-devel x86_64 2.25-12.fc26 updates 966 k glibc-headers x86_64 2.25-12.fc26 updates 516 k groff-base x86_64 1.22.3-9.fc26 fedora 1.0 M guile x86_64 5:2.0.14-1.fc26 fedora 3.5 M isl x86_64 0.16.1-1.fc26 fedora 821 k jsoncpp x86_64 1.8.1-3.fc26 updates 83 k kernel-headers x86_64 4.13.9-200.fc26 updates 1.2 M libatomic_ops x86_64 7.4.4-2.fc26 fedora 31 k libgomp x86_64 7.2.1-2.fc26 updates 197 k libmpc x86_64 1.0.2-6.fc26 fedora 55 k libstdc++ x86_64 7.2.1-2.fc26 updates 462 k libstdc++-devel x86_64 7.2.1-2.fc26 updates 1.9 M libtool-ltdl x86_64 2.4.6-17.fc26 fedora 55 k libuv x86_64 1:1.11.0-1.fc26 fedora 112 k m4 x86_64 1.4.18-3.fc26 fedora 217 k openssl x86_64 1:1.1.0f-7.fc26 updates 543 k perl x86_64 4:5.24.3-395.fc26 updates 6.1 M perl-Carp noarch 1.40-366.fc26 fedora 28 k perl-Data-Dumper x86_64 2.161-2.fc26 fedora 54 k perl-Encode x86_64 4:2.88-6.fc26 fedora 1.5 M perl-Errno x86_64 1.25-395.fc26 updates 69 k perl-Exporter noarch 5.72-367.fc26 fedora 32 k perl-File-Path noarch 2.12-367.fc26 fedora 34 k perl-File-Temp noarch 0.230.400-2.fc26 fedora 61 k perl-Getopt-Long noarch 2.49.1-2.fc26 fedora 61 k perl-HTTP-Tiny noarch 0.070-2.fc26 fedora 56 k perl-IO x86_64 1.36-395.fc26 updates 134 k perl-MIME-Base64 x86_64 3.15-366.fc26 fedora 28 k perl-Net-SSLeay x86_64 1.81-1.fc26 fedora 333 k perl-PathTools x86_64 3.63-367.fc26 fedora 87 k perl-Pod-Escapes noarch 1:1.07-366.fc26 fedora 19 k perl-Pod-Perldoc noarch 3.28-1.fc26 fedora 86 k perl-Pod-Simple noarch 1:3.35-2.fc26 fedora 211 k perl-Pod-Usage noarch 4:1.69-2.fc26 fedora 33 k perl-Scalar-List-Utils x86_64 3:1.48-1.fc26 updates 65 k perl-Socket x86_64 4:2.024-2.fc26 fedora 55 k perl-Term-ANSIColor noarch 4.06-2.fc26 fedora 44 k perl-Term-Cap noarch 1.17-366.fc26 fedora 21 k perl-Text-ParseWords noarch 3.30-366.fc26 fedora 16 k perl-Text-Tabs+Wrap noarch 2013.0523-366.fc26 fedora 22 k perl-Thread-Queue noarch 3.12-1.fc26 fedora 22 k perl-Time-Local noarch 1:1.250-2.fc26 fedora 30 k perl-URI noarch 1.71-6.fc26 fedora 115 k perl-Unicode-Normalize x86_64 1.25-366.fc26 fedora 79 k perl-constant noarch 1.33-368.fc26 fedora 23 k perl-libnet noarch 3.10-3.fc26 fedora 119 k perl-libs x86_64 4:5.24.3-395.fc26 updates 1.5 M perl-macros x86_64 4:5.24.3-395.fc26 updates 65 k perl-parent noarch 1:0.236-2.fc26 fedora 18 k perl-podlators noarch 4.09-2.fc26 fedora 114 k perl-threads x86_64 1:2.16-1.fc26 fedora 58 k perl-threads-shared x86_64 1.57-1.fc26 fedora 45 k rhash x86_64 1.3.4-2.fc26 fedora 125 k Installing weak dependencies: perl-IO-Socket-IP noarch 0.39-1.fc26 fedora 45 k perl-IO-Socket-SSL noarch 2.049-1.fc26 fedora 270 k perl-Mozilla-CA noarch 20160104-4.fc26 fedora 13 k Transaction Summary ================================================================================ Install 66 Packages Upgrade 5 Packages Total download size: 80 M Downloading Packages: (1/71): make-4.2.1-2.fc26.x86_64.rpm 683 kB/s | 492 kB 00:00 (2/71): perl-Carp-1.40-366.fc26.noarch.rpm 371 kB/s | 28 kB 00:00 (3/71): perl-Exporter-5.72-367.fc26.noarch.rpm 419 kB/s | 32 kB 00:00 (4/71): autoconf-2.69-24.fc26.noarch.rpm 798 kB/s | 709 kB 00:00 (5/71): automake-1.15-9.fc26.noarch.rpm 728 kB/s | 695 kB 00:00 (6/71): perl-File-Path-2.12-367.fc26.noarch.rpm 401 kB/s | 34 kB 00:00 (7/71): perl-Getopt-Long-2.49.1-2.fc26.noarch.r 443 kB/s | 61 kB 00:00 (8/71): perl-PathTools-3.63-367.fc26.x86_64.rpm 915 kB/s | 87 kB 00:00 (9/71): perl-Thread-Queue-3.12-1.fc26.noarch.rp 161 kB/s | 22 kB 00:00 (10/71): perl-constant-1.33-368.fc26.noarch.rpm 248 kB/s | 23 kB 00:00 (11/71): perl-threads-2.16-1.fc26.x86_64.rpm 450 kB/s | 58 kB 00:00 (12/71): gc-7.6.0-2.fc26.x86_64.rpm 612 kB/s | 108 kB 00:00 (13/71): perl-Data-Dumper-2.161-2.fc26.x86_64.r 381 kB/s | 54 kB 00:00 (14/71): m4-1.4.18-3.fc26.x86_64.rpm 891 kB/s | 217 kB 00:00 (15/71): perl-Text-ParseWords-3.30-366.fc26.noa 217 kB/s | 16 kB 00:00 (16/71): perl-Pod-Usage-1.69-2.fc26.noarch.rpm 442 kB/s | 33 kB 00:00 (17/71): perl-threads-shared-1.57-1.fc26.x86_64 602 kB/s | 45 kB 00:00 (18/71): libatomic_ops-7.4.4-2.fc26.x86_64.rpm 236 kB/s | 31 kB 00:00 (19/71): libtool-ltdl-2.4.6-17.fc26.x86_64.rpm 428 kB/s | 55 kB 00:00 (20/71): perl-Pod-Perldoc-3.28-1.fc26.noarch.rp 1.1 MB/s | 86 kB 00:00 (21/71): perl-podlators-4.09-2.fc26.noarch.rpm 785 kB/s | 114 kB 00:00 (22/71): perl-File-Temp-0.230.400-2.fc26.noarch 821 kB/s | 61 kB 00:00 (23/71): perl-HTTP-Tiny-0.070-2.fc26.noarch.rpm 729 kB/s | 56 kB 00:00 (24/71): perl-Pod-Simple-3.35-2.fc26.noarch.rpm 940 kB/s | 211 kB 00:00 (25/71): perl-parent-0.236-2.fc26.noarch.rpm 240 kB/s | 18 kB 00:00 (26/71): perl-Term-ANSIColor-4.06-2.fc26.noarch 584 kB/s | 44 kB 00:00 (27/71): groff-base-1.22.3-9.fc26.x86_64.rpm 1.4 MB/s | 1.0 MB 00:00 (28/71): perl-Term-Cap-1.17-366.fc26.noarch.rpm 206 kB/s | 21 kB 00:00 (29/71): perl-MIME-Base64-3.15-366.fc26.x86_64. 375 kB/s | 28 kB 00:00 (30/71): perl-Socket-2.024-2.fc26.x86_64.rpm 718 kB/s | 55 kB 00:00 (31/71): perl-Time-Local-1.250-2.fc26.noarch.rp 405 kB/s | 30 kB 00:00 (32/71): perl-Pod-Escapes-1.07-366.fc26.noarch. 254 kB/s | 19 kB 00:00 (33/71): perl-Text-Tabs+Wrap-2013.0523-366.fc26 305 kB/s | 22 kB 00:00 (34/71): guile-2.0.14-1.fc26.x86_64.rpm 1.9 MB/s | 3.5 MB 00:01 (35/71): cmake-filesystem-3.9.1-1.fc26.x86_64.r 653 kB/s | 35 kB 00:00 (36/71): cmake-data-3.9.1-1.fc26.noarch.rpm 2.3 MB/s | 1.2 MB 00:00 (37/71): libuv-1.11.0-1.fc26.x86_64.rpm 307 kB/s | 112 kB 00:00 (38/71): rhash-1.3.4-2.fc26.x86_64.rpm 324 kB/s | 125 kB 00:00 (39/71): cmake-3.9.1-1.fc26.x86_64.rpm 2.6 MB/s | 7.4 MB 00:02 (40/71): isl-0.16.1-1.fc26.x86_64.rpm 1.1 MB/s | 821 kB 00:00 (41/71): libmpc-1.0.2-6.fc26.x86_64.rpm 684 kB/s | 55 kB 00:00 (42/71): cpp-7.2.1-2.fc26.x86_64.rpm 2.5 MB/s | 9.2 MB 00:03 (43/71): glibc-devel-2.25-12.fc26.x86_64.rpm 2.1 MB/s | 966 kB 00:00 (44/71): glibc-headers-2.25-12.fc26.x86_64.rpm 2.4 MB/s | 516 kB 00:00 (45/71): perl-Encode-2.88-6.fc26.x86_64.rpm 1.3 MB/s | 1.5 MB 00:01 (46/71): perl-Scalar-List-Utils-1.48-1.fc26.x86 1.7 MB/s | 65 kB 00:00 (47/71): gcc-7.2.1-2.fc26.x86_64.rpm 3.5 MB/s | 21 MB 00:06 (48/71): perl-Errno-1.25-395.fc26.x86_64.rpm 1.2 MB/s | 69 kB 00:00 (49/71): perl-libs-5.24.3-395.fc26.x86_64.rpm 3.0 MB/s | 1.5 MB 00:00 (50/71): perl-Unicode-Normalize-1.25-366.fc26.x 546 kB/s | 79 kB 00:00 (51/71): perl-IO-1.36-395.fc26.x86_64.rpm 2.2 MB/s | 134 kB 00:00 (52/71): libstdc++-7.2.1-2.fc26.x86_64.rpm 1.7 MB/s | 462 kB 00:00 (53/71): perl-macros-5.24.3-395.fc26.x86_64.rpm 1.5 MB/s | 65 kB 00:00 (54/71): gcc-c++-7.2.1-2.fc26.x86_64.rpm 2.8 MB/s | 11 MB 00:03 (55/71): kernel-headers-4.13.9-200.fc26.x86_64. 2.4 MB/s | 1.2 MB 00:00 (56/71): libgomp-7.2.1-2.fc26.x86_64.rpm 2.6 MB/s | 197 kB 00:00 (57/71): jsoncpp-1.8.1-3.fc26.x86_64.rpm 2.1 MB/s | 83 kB 00:00 (58/71): libstdc++-devel-7.2.1-2.fc26.x86_64.rp 2.8 MB/s | 1.9 MB 00:00 (59/71): perl-Mozilla-CA-20160104-4.fc26.noarch 59 kB/s | 13 kB 00:00 (60/71): perl-IO-Socket-IP-0.39-1.fc26.noarch.r 120 kB/s | 45 kB 00:00 (61/71): perl-IO-Socket-SSL-2.049-1.fc26.noarch 723 kB/s | 270 kB 00:00 (62/71): openssl-1.1.0f-7.fc26.x86_64.rpm 2.9 MB/s | 543 kB 00:00 (63/71): perl-Net-SSLeay-1.81-1.fc26.x86_64.rpm 1.2 MB/s | 333 kB 00:00 (64/71): perl-URI-1.71-6.fc26.noarch.rpm 724 kB/s | 115 kB 00:00 (65/71): perl-5.24.3-395.fc26.x86_64.rpm 2.8 MB/s | 6.1 MB 00:02 (66/71): libgcc-7.2.1-2.fc26.x86_64.rpm 1.5 MB/s | 90 kB 00:00 (67/71): perl-libnet-3.10-3.fc26.noarch.rpm 1.2 MB/s | 119 kB 00:00 (68/71): glibc-langpack-en-2.25-12.fc26.x86_64. 2.6 MB/s | 291 kB 00:00 (69/71): libcrypt-nss-2.25-12.fc26.x86_64.rpm 2.0 MB/s | 54 kB 00:00 (70/71): glibc-common-2.25-12.fc26.x86_64.rpm 1.5 MB/s | 890 kB 00:00 (71/71): glibc-2.25-12.fc26.x86_64.rpm 3.1 MB/s | 3.4 MB 00:01 -------------------------------------------------------------------------------- Total 6.0 MB/s | 80 MB 00:13 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Upgrading : glibc-common-2.25-12.fc26.x86_64 1/76 Upgrading : glibc-langpack-en-2.25-12.fc26.x86_64 2/76 Running scriptlet: glibc-2.25-12.fc26.x86_64 3/76 Upgrading : glibc-2.25-12.fc26.x86_64 3/76 Running scriptlet: glibc-2.25-12.fc26.x86_64 3/76 Upgrading : libgcc-7.2.1-2.fc26.x86_64 4/76 Running scriptlet: libgcc-7.2.1-2.fc26.x86_64 4/76 Installing : libstdc++-7.2.1-2.fc26.x86_64 5/76 Running scriptlet: libstdc++-7.2.1-2.fc26.x86_64 5/76 Upgrading : libcrypt-nss-2.25-12.fc26.x86_64 6/76 Running scriptlet: libcrypt-nss-2.25-12.fc26.x86_64 6/76 Installing : perl-Exporter-5.72-367.fc26.noarch 7/76 Installing : perl-libs-4:5.24.3-395.fc26.x86_64 8/76 Running scriptlet: perl-libs-4:5.24.3-395.fc26.x86_64 8/76 Installing : perl-Carp-1.40-366.fc26.noarch 9/76 Installing : perl-Scalar-List-Utils-3:1.48-1.fc26.x86_64 10/76 Installing : perl-Text-ParseWords-3.30-366.fc26.noarch 11/76 Installing : libmpc-1.0.2-6.fc26.x86_64 12/76 Running scriptlet: libmpc-1.0.2-6.fc26.x86_64 12/76 Installing : cmake-filesystem-3.9.1-1.fc26.x86_64 13/76 Installing : cpp-7.2.1-2.fc26.x86_64 14/76 Running scriptlet: cpp-7.2.1-2.fc26.x86_64 14/76 Installing : perl-Term-ANSIColor-4.06-2.fc26.noarch 15/76 Installing : perl-macros-4:5.24.3-395.fc26.x86_64 16/76 Installing : perl-Socket-4:2.024-2.fc26.x86_64 17/76 Installing : perl-Errno-1.25-395.fc26.x86_64 18/76 Installing : perl-File-Path-2.12-367.fc26.noarch 19/76 Installing : perl-PathTools-3.63-367.fc26.x86_64 20/76 Installing : perl-constant-1.33-368.fc26.noarch 21/76 Installing : perl-threads-shared-1.57-1.fc26.x86_64 22/76 Installing : perl-threads-1:2.16-1.fc26.x86_64 23/76 Installing : perl-parent-1:0.236-2.fc26.noarch 24/76 Installing : perl-Text-Tabs+Wrap-2013.0523-366.fc26.noarch 25/76 Installing : perl-Unicode-Normalize-1.25-366.fc26.x86_64 26/76 Installing : perl-IO-1.36-395.fc26.x86_64 27/76 Installing : perl-4:5.24.3-395.fc26.x86_64 28/76 Installing : perl-MIME-Base64-3.15-366.fc26.x86_64 29/76 Installing : perl-Data-Dumper-2.161-2.fc26.x86_64 30/76 Installing : perl-IO-Socket-IP-0.39-1.fc26.noarch 31/76 Installing : perl-Time-Local-1:1.250-2.fc26.noarch 32/76 Installing : perl-HTTP-Tiny-0.070-2.fc26.noarch 33/76 Installing : perl-libnet-3.10-3.fc26.noarch 34/76 Installing : perl-Net-SSLeay-1.81-1.fc26.x86_64 35/76 Installing : perl-File-Temp-0.230.400-2.fc26.noarch 36/76 Installing : perl-Term-Cap-1.17-366.fc26.noarch 37/76 Installing : perl-Pod-Escapes-1:1.07-366.fc26.noarch 38/76 Installing : perl-Thread-Queue-3.12-1.fc26.noarch 39/76 Installing : groff-base-1.22.3-9.fc26.x86_64 40/76 Installing : perl-Pod-Simple-1:3.35-2.fc26.noarch 41/76 Installing : perl-Encode-4:2.88-6.fc26.x86_64 42/76 Installing : perl-Getopt-Long-2.49.1-2.fc26.noarch 43/76 Installing : perl-podlators-4.09-2.fc26.noarch 44/76 Installing : perl-Pod-Perldoc-3.28-1.fc26.noarch 45/76 Installing : perl-Pod-Usage-4:1.69-2.fc26.noarch 46/76 Installing : perl-URI-1.71-6.fc26.noarch 47/76 Installing : libstdc++-devel-7.2.1-2.fc26.x86_64 48/76 Installing : jsoncpp-1.8.1-3.fc26.x86_64 49/76 Running scriptlet: jsoncpp-1.8.1-3.fc26.x86_64 49/76 Installing : m4-1.4.18-3.fc26.x86_64 50/76 Running scriptlet: m4-1.4.18-3.fc26.x86_64 50/76 Installing : autoconf-2.69-24.fc26.noarch 51/76 Running scriptlet: autoconf-2.69-24.fc26.noarch 51/76 install-info: No such file or directory for /usr/share/info/autoconf.info Installing : libatomic_ops-7.4.4-2.fc26.x86_64 52/76 Running scriptlet: libatomic_ops-7.4.4-2.fc26.x86_64 52/76 Installing : gc-7.6.0-2.fc26.x86_64 53/76 Running scriptlet: gc-7.6.0-2.fc26.x86_64 53/76 Installing : libtool-ltdl-2.4.6-17.fc26.x86_64 54/76 Running scriptlet: libtool-ltdl-2.4.6-17.fc26.x86_64 54/76 Installing : guile-5:2.0.14-1.fc26.x86_64 55/76 Running scriptlet: guile-5:2.0.14-1.fc26.x86_64 55/76 Installing : make-1:4.2.1-2.fc26.x86_64 56/76 Running scriptlet: make-1:4.2.1-2.fc26.x86_64 56/76 Installing : openssl-1:1.1.0f-7.fc26.x86_64 57/76 Installing : libuv-1:1.11.0-1.fc26.x86_64 58/76 Running scriptlet: libuv-1:1.11.0-1.fc26.x86_64 58/76 Installing : rhash-1.3.4-2.fc26.x86_64 59/76 Running scriptlet: rhash-1.3.4-2.fc26.x86_64 59/76 Installing : cmake-data-3.9.1-1.fc26.noarch 60/76 Installing : cmake-3.9.1-1.fc26.x86_64 61/76 Installing : isl-0.16.1-1.fc26.x86_64 62/76 Running scriptlet: isl-0.16.1-1.fc26.x86_64 62/76 Installing : libgomp-7.2.1-2.fc26.x86_64 63/76 Running scriptlet: libgomp-7.2.1-2.fc26.x86_64 63/76 Installing : kernel-headers-4.13.9-200.fc26.x86_64 64/76 Running scriptlet: glibc-headers-2.25-12.fc26.x86_64 65/76 Installing : glibc-headers-2.25-12.fc26.x86_64 65/76 Installing : glibc-devel-2.25-12.fc26.x86_64 66/76 Running scriptlet: glibc-devel-2.25-12.fc26.x86_64 66/76 Installing : gcc-7.2.1-2.fc26.x86_64 67/76 Running scriptlet: gcc-7.2.1-2.fc26.x86_64 67/76 Installing : gcc-c++-7.2.1-2.fc26.x86_64 68/76 Installing : perl-IO-Socket-SSL-2.049-1.fc26.noarch 69/76 Installing : automake-1.15-9.fc26.noarch 70/76 Running scriptlet: automake-1.15-9.fc26.noarch 70/76 install-info: No such file or directory for /usr/share/info/automake.info.gz Installing : perl-Mozilla-CA-20160104-4.fc26.noarch 71/76 Cleanup : libcrypt-nss-2.25-10.fc26.x86_64 72/76 Running scriptlet: libcrypt-nss-2.25-10.fc26.x86_64 72/76 Cleanup : glibc-common-2.25-10.fc26.x86_64 73/76 Cleanup : glibc-langpack-en-2.25-10.fc26.x86_64 74/76 Cleanup : glibc-2.25-10.fc26.x86_64 75/76 Running scriptlet: glibc-2.25-10.fc26.x86_64 75/76 Cleanup : libgcc-7.1.1-3.fc26.x86_64 76/76 Running scriptlet: libgcc-7.1.1-3.fc26.x86_64 76/76 Running scriptlet: guile-5:2.0.14-1.fc26.x86_64 76/76BDB1539 Build signature doesn't match environment failed loading RPMDB The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'. The command '/bin/sh -c dnf -y install automake cmake gcc gcc-c++ glibc make' returned a non-zero code: 1 $ docker image inspect fedora:26 [ { "Id": "sha256:b0b140824a486ccc0f7968f3c6ceb6982b4b77e82ef8b4faaf2806049fc266df", "RepoTags": [ "fedora:26" ], "RepoDigests": [ "fedora@sha256:613830d906f8897c05fd737aea8c7c00e4ecb5a518eedf71958c2c110532957c" ], "Parent": "", "Comment": "", "Created": "2017-09-15T23:28:22.816153183Z", "Container": "325c38d9922c257e0ccfec8503e61a156fa6ddbba9633f33b3e8acdf19d1307a", "ContainerConfig": { "Hostname": "325c38d9922c", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "DISTTAG=f26container", "FGC=f26", "FBR=f26" ], "Cmd": [ "/bin/sh", "-c", "#(nop) ADD file:6fef8af91ffd240db5c0e394621757fc453e5f7312e33f04d98d04fef009b078 in / " ], "Image": "sha256:b01a4b5daeddf89b88f00abe8271e0e10ff1d520d227f91ce9af5d10dc94aaf1", "Volumes": null, "WorkingDir": "", "Entrypoint": null, "OnBuild": null, "Labels": {} }, "DockerVersion": "17.06.2-ce", "Author": "[Adam Miller <maxamillion>] [Patrick Uiterwijk <patrick>]", "Config": { "Hostname": "", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "DISTTAG=f26container", "FGC=f26", "FBR=f26" ], "Cmd": null, "Image": "sha256:b01a4b5daeddf89b88f00abe8271e0e10ff1d520d227f91ce9af5d10dc94aaf1", "Volumes": null, "WorkingDir": "", "Entrypoint": null, "OnBuild": null, "Labels": null }, "Architecture": "amd64", "Os": "linux", "Size": 231079876, "VirtualSize": 231079876, "GraphDriver": { "Data": { "MergedDir": "/var/lib/docker/overlay2/ab00f10bc15af977ffb201fd3ae1d01d41ce2f5b9aad91970c9e32ab392b2455/merged", "UpperDir": "/var/lib/docker/overlay2/ab00f10bc15af977ffb201fd3ae1d01d41ce2f5b9aad91970c9e32ab392b2455/diff", "WorkDir": "/var/lib/docker/overlay2/ab00f10bc15af977ffb201fd3ae1d01d41ce2f5b9aad91970c9e32ab392b2455/work" }, "Name": "overlay2" }, "RootFS": { "Type": "layers", "Layers": [ "sha256:72e455bdda27d1fad7d0b6e4e2b7b10893e90576a5008e12af9bdaea0f926b1d" ] }, "Metadata": { "LastTagTime": "0001-01-01T00:00:00Z" } } ] $ uname -a Linux linuxbox 4.13.9-200.fc26.x86_64 #1 SMP Mon Oct 23 13:52:45 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ docker --version Docker version 17.11.0-ce-rc2, build d7062e5
fedora:26 at docker hub is out of date with respect to the one from fedoraproject ``` registry.fedoraproject.org/fedora 26 ba857ff07853 2 weeks ago 231MB fedora 26 b0b140824a48 6 weeks ago 231MB ``` Will dockerhub's "Author": "[Adam Miller <maxamillion>] [Patrick Uiterwijk <patrick>]", please update it?
(In reply to Mark Hirota from comment #76) > (In reply to Panu Matilainen from comment #75) > > > So before complaining about not working, everybody PLEASE make sure you have > > the updated libsolv version. Oh and there's a related issue on rpm-side too > > (there's an update pending stable), but hitting that would manifest in a > > different way. > > Hi, I'm not sure what you mean "make sure you have the updated libsolv > version"? I tried both in the host environment (which happens to be F26) > and inside the docker image build (Dockerfile) -- but it still fails. I am > using the base docker fedora:26 image from hub.docker.com (Please see my > config below). Inside the docker. As in: [root@sopuli ~]# docker run --rm -ti fedora:26 bash [root@34c95cb831ca /]# rpm -q libsolv libsolv-0.6.29-2.fc26.x86_64 If libsolv is older than that, you need to first update libsolv in a separate transaction, ie (inside docker still): # dnf update libsolv [...] # dnf update With that, you'll still get "BDB1539 Build signature doesn't match environment" but not "failed loading RPMDB" and the operation actually succeeds. There's no indication of the libsolv version inside docker so impossible to say anything except that it's behaving exactly like the version without the workaround. But there really should be a diagnostic message for the workaround, sigh...
(In reply to Panu Matilainen from comment #78) > There's no indication of the libsolv version inside docker > so impossible to say anything except that it's behaving exactly like > the version without the workaround. That was supposed to say: There's no indication of the libsolv version inside docker in the log you included in comment #76 [...]
Created attachment 1348517 [details] libsolv patch to fall back to DB_PRIVATE on DB_VERSION_MISMATCH Updated version of the patch, now including a diagnostic message + a better explanation of what the patch is about.
libsolv-0.6.30-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-9d9767cf8a
libsolv-0.6.30-2.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-f8c708fa8d
libsolv-0.6.30-2.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-f8c708fa8d
libsolv-0.6.30-2.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-9d9767cf8a
I have requested the container to be rebuilt here: https://pagure.io/releng/issue/7125
libsolv-0.6.30-2.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
libsolv-0.6.30-2.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.