Description of problem: kernel-debuginfo is marked as one of the "installonlypkg(kernel)" packages, so dnf will install updates in parallel instead of replacing it. However, the tools built in the scripts/ directory (and shipped in kernel-devel) tend to have identical build-ids, so the /usr/lib/debug/.build-id/... symlinks cause rpm conflicts. Version-Release number of selected component (if applicable): kernel-debuginfo-4.6.6-300.fc24.x86_64 kernel-debuginfo-4.6.7-300.fc24.x86_64 How reproducible: 100% Steps to Reproduce: 1. Install kernel-debuginfo-4.6.6-300.fc24.x86_64 2. Try to update/install to kernel-debuginfo-4.6.7-300.fc24.x86_64 Actual results: Error: Transaction check error: file /usr/lib/debug/.build-id/0e/1e7f3c4cbc515b3644d2f4638a3c75665e62fa.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/0e/1e7f3c4cbc515b3644d2f4638a3c75665e62fa.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/17/fe050d200b300a757af9de5cb8538785687fad.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/17/fe050d200b300a757af9de5cb8538785687fad.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/49/73048b661120f90d55b921e9f5cc9761a8a23a.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/49/73048b661120f90d55b921e9f5cc9761a8a23a.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/63/e52f339e692016655925e8e333bc2e3723c089.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/63/e52f339e692016655925e8e333bc2e3723c089.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/6f/cc1f986c0a4c474d5edcbb0a7a1a32476b46e0.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/6f/cc1f986c0a4c474d5edcbb0a7a1a32476b46e0.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/80/f8db17b84f8f325ece71d055c0a27268807b31.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/80/f8db17b84f8f325ece71d055c0a27268807b31.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/84/2924caf84d88dd2a262e97db45f97836a449bf.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/84/2924caf84d88dd2a262e97db45f97836a449bf.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/8f/2e01e15a5d5e0fc8c02a61e5aaf737b5e77b32.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/8f/2e01e15a5d5e0fc8c02a61e5aaf737b5e77b32.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/a6/51e3b2a24f537a9a7e48a2359c5a54c26d34e5.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/a6/51e3b2a24f537a9a7e48a2359c5a54c26d34e5.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/aa/a11f45dc3b336b7e50a883f9e6b8414c4003fe.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/aa/a11f45dc3b336b7e50a883f9e6b8414c4003fe.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/b6/2f650dbf6f96bad3f2918e4510985ff0396cf4.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/b6/2f650dbf6f96bad3f2918e4510985ff0396cf4.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/bd/3bedfebcc38472af59da7510a71f2afdb54824.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/bd/3bedfebcc38472af59da7510a71f2afdb54824.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/c7/5ac79e4d19045ecda07b8b98b9a18387ce85f7.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/c7/5ac79e4d19045ecda07b8b98b9a18387ce85f7.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/d2/f8344ba322af47f3ba89b620a69d1068b983fb.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/d2/f8344ba322af47f3ba89b620a69d1068b983fb.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/d3/0a6567cc1719dc853106070c6c4e8fd641d8ce.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/d3/0a6567cc1719dc853106070c6c4e8fd641d8ce.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/f6/51b12b8eea84660bdac09b87fd43c38be81d43.1 from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 file /usr/lib/debug/.build-id/f6/51b12b8eea84660bdac09b87fd43c38be81d43.1.debug from install of kernel-debuginfo-4.6.7-300.fc24.x86_64 conflicts with file from package kernel-debuginfo-4.6.6-300.fc24.x86_64 (those are all symlinks to something in scripts/) Expected results: No conflicts Additional info: The files in question actually don't have any .debug* sections at all, which may be why they didn't get unique build-ids. Usually the source paths in debuginfo are enough to distinguish files, but otherwise these scripts/ tools are mostly unchanged. So one approach might be to just filter scripts/ from debuginfo processing. Better would to actually give them debuginfo. These are compiled using HOSTCFLAGS, which does not include "-g". This should be added, or even add the entire rpm %{optflags}. Also of note, I actually do have two kernel-debuginfo packages installed already, for 4.6.5 and 4.6.6. But note that the conflicts above are all suffixed ".1.debug", whereas the equivalent files in 4.6.5 are simply ".debug". That extra ".1" comes from find_debuginfo.sh when it sees duplicate files within the same build -- make_id_link() calls make_id_dup_link(), then prints "*** WARNING: identical binaries are copied, not linked:" and the file names. In 4.6.5's x86_64 build.log, you can see it warned that each +debug/scripts/ was identical to the base /scripts/ tools, which means the +debug links would get the ".1.debug" treatment. In 4.6.6 and 4.6.7, it warned the other way around, so the base links got ".1.debug". That's how I didn't get conflicts between 4.6.5 and 4.6.6, but this is surely luck of the draw in find-debuginfo's order.
I think this might be solved by the following new feature in rpm git (not yet released): Make it possible to have unique build-ids across build versions/releases. http://rpm.org/gitweb?p=rpm.git;a=commit;h=5ef1166ad96e3545784fa5420a49e1b2cd481e8e
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 24 kernel bugs. Fedora 24 has now been rebased to 4.7.4-200.fc24. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 25, and are still experiencing this issue, please change the version to Fedora 25. If you experience different issues, please open a new bug report for those.
I just ran into this again updating from kernel-debuginfo-4.7.3-200.fc24.x86_64 to kernel-debuginfo-4.7.4-200.fc24.x86_64. Several /usr/lib/debug/.build-id/... symlinks conflict between these, all pointing to scripts/ binaries.
Note that the rpm feature mentioned in comment #3 has been integrated upstream, but is not yet in a release. When it is then the distro has to set the _unique_build_ids macro to activate it for new builds. That should resolve the issue. The tracking bug for parallel installable debuginfo is bug #1340819.
Or without depending on rpm, HOSTCFLAGS just needs %{optflags}, especially -g. The kernel's build path includes the full verrel, which shows up in debuginfo in entries like DW_AT_comp_dir, and we get a unique build-id. That's why *most* of the kernel binaries are fine, just not in scripts/ where they lack -g.
(In reply to Josh Stone from comment #6) > Or without depending on rpm, HOSTCFLAGS just needs %{optflags}, especially > -g. > > The kernel's build path includes the full verrel, which shows up in > debuginfo in entries like DW_AT_comp_dir, and we get a unique build-id. > That's why *most* of the kernel binaries are fine, just not in scripts/ > where they lack -g. That is interesting. I hadn't thought of the no-debuginfo case. Another change in upstream rpm is a new macro to move the none-debug build-id symlinks into the main package. In which case I presume we get actual conflicts in the kernel package itself. Not good. This definitely needs some testing against the kernel package before enabling. Could you post where the build-id symlinks point to? What is the reason for not including -g when building these binaries? Also why does the build-id change when building with -g? In theory (unless there are bugs in gcc) that should produce the exact same executable sections. Just with extra non-allocated .debug sections. Which should not change the calculated build-id. The new rpm debugedit feature to update the build-id won't trigger for executables without debuginfo. So that will indeed not help in this case.
(In reply to Mark Wielaard from comment #7) > Could you post where the build-id symlinks point to? For instance: /usr/lib/debug/.build-id/33/a569cbd286744852ae531e3feaaa1dc1887108 -> ../../../../src/kernels/4.7.4-200.fc24.x86_64/scripts/unifdef /usr/lib/debug/.build-id/33/a569cbd286744852ae531e3feaaa1dc1887108.debug -> ../../usr/src/kernels/4.7.4-200.fc24.x86_64/scripts/unifdef.debug Note that unifdef.debug does exist, but it has no .debug* sections. These exact same .build-id symlinks existed in my prior kernel-debuginfo, but pointing to a 4.7.3 path, so that's the conflict. > What is the reason for not including -g when building these binaries? I'm sure that's just an oversight. The base Makefile has hard-coded HOSTCFLAGS, and nothing in kernel.spec tries to override it. > Also why does the build-id change when building with -g? In theory (unless > there are bugs in gcc) that should produce the exact same executable > sections. Just with extra non-allocated .debug sections. Which should not > change the calculated build-id. There's a deeper analysis in the related bug I linked in comment #1. Sorry that's all private, but I don't want to be the one to leak info. Maybe we should ask for permission to copy that here in the open. AIUI, the recomputed build-id of "debugedit -i" is usually affected by the DW_AT_comp_dir, especially since that gets rewritten to a /usr/src/debug/ path.
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There are a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 24 kernel bugs. Fedora 25 has now been rebased to 4.10.9-100.fc24. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 26, and are still experiencing this issue, please change the version to Fedora 26. If you experience different issues, please open a new bug report for those.
*********** MASS BUG UPDATE ************** This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 2 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.
Mark, you're still working on these debuginfo packaging changes, right?
(In reply to Josh Stone from comment #11) > Mark, you're still working on these debuginfo packaging changes, right? Yes. But first we are trying to get the debuginfo package generation cleaned up to use less magic in the kernel.spec. For now that is with unique build-ids turned off. See this thread: https://lists.fedoraproject.org/archives/list/kernel@lists.fedoraproject.org/thread/LCXXB6EBVUZRJ4KQ26ZSTNTBDXGAAO72/
This message is a reminder that Fedora 24 is nearing its end of life. Approximately 2 (two) weeks from now Fedora will stop maintaining and issuing updates for Fedora 24. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '24'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 24 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 24 changed to end-of-life (EOL) status on 2017-08-08. Fedora 24 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.