Is this more fallout from https://bugzilla.redhat.com/show_bug.cgi?id=1430587

*** Bug 1431896 has been marked as a duplicate of this bug. ***

I'm pretty sure this is not kernel specific but a general RPM issue.

This is basically same as https://bugzilla.redhat.com/show_bug.cgi?id=1432372, so kernel needs to be rebuilt...

But thinking about upgrade path.. not sure how to properly handle that. Probably simple:

# find /usr/lib/.build-id -type d -exec chmod 755 {} ';'

Laura, if you will find more such cases and solution from above works, probably we should announce it on devel@.

*** Bug 1433169 has been marked as a duplicate of this bug. ***

Igor: remember, kernel packages are a special case. We do not *upgrade* kernels. We parallel install them. So kernel packages cannot contain files with the same name but different contents. If rpm is causing this to happen, that's a problem and needs to be fixed.

(In reply to Adam Williamson from comment #6)
> Igor: remember, kernel packages are a special case. We do not *upgrade*
> kernels. We parallel install them. So kernel packages cannot contain files
> with the same name but different contents. If rpm is causing this to happen,
> that's a problem and needs to be fixed.
Yes, I know. But problem is that broken RPM produced wrong packages. And we can't do anything about it now. We can't disable check or fix user systems automatically.

Erm..., the issue here is not about file attribute differences produced by a broken rpm. Its about certain elf objects in the kernel tree having the same exact build-id between two different kernel packages leading to /usr/lib/.build-id/...<symlink> conflicts between the two.
These symlinks are actually different and not just having different attributes.


In my case it was the vdso/vdso{32,64}.so -s from 
kernel-core-4.11.0-0.rc2.git0.1.fc27.x86_64 and kernel-core-4.11.0-0.rc2.git1.2.fc27.x86_64

I checked the .so-s separately from rpm and their build-id's were the same. So I can't fault rpm for having the conflicting symlink

How is rpm supposed to deal with that? and why these .so-s had the same build-id in the first place ?

(In reply to Adam Williamson from comment #6)
> Igor: remember, kernel packages are a special case. We do not *upgrade*
> kernels. We parallel install them. So kernel packages cannot contain files
> with the same name but different contents. If rpm is causing this to happen,
> that's a problem and needs to be fixed.

Thanks for the analysis. That is exactly what is happening and what rpm should be preventing. Different builds should generate generate different build-ids. Which is what rpm tries to guarantee.

But the kernel package defeats rpm by calling debugedit directly and then overriding __debug_install_post and how find-debuginfo.sh is called.

The correct fix is for the kernel to call find-debuginfo.sh and debugedit like how rpm would. That means adding --ver-rel "%{VERSION}-%{RELEASE}" to the debuginfo_args and changing the debugedit call to include --build-id-seed=--ver-rel "%{VERSION}-%{RELEASE}". This is all a little involved because the kernel spec makes various assumptions about the precise workings of debugedit and find-debuginfo.sh and calls them (multiple times!) in different order.

So the easiest fix is reverting to the old setting for build-id links for the kernel by setting as follows:

diff --git a/kernel.spec b/kernel.spec
index 9a5fe0b..9d7f7ba 100644
--- a/kernel.spec
+++ b/kernel.spec
@@ -183,6 +183,9 @@ Summary: The Linux kernel
 %define _enable_debug_packages 0
 %endif
 %define debuginfodir /usr/lib/debug
+# Needed because we override almost everything involving build-ids
+# and debuginfo generation. Currently we rely on the old alldebug setting.
+%global _build_id_links alldebug
 
 # kernel PAE is only built on i686 and ARMv7.
 %ifnarch i686 armv7hl

But if you do please don't close this bug yet. We should come up with a better way for the kernel package to do this that doesn't require so many hacks overriding rpm macros to generate correct buildids and debuginfo. It also prevents the kernel to take advantage of some pending upstream improvements to rpm for parallel debuginfo generation for example.

(In reply to Mark Wielaard from comment #9)
> (In reply to Adam Williamson from comment #6)
> > Igor: remember, kernel packages are a special case. We do not *upgrade*
> > kernels. We parallel install them. So kernel packages cannot contain files
> > with the same name but different contents. If rpm is causing this to happen,
> > that's a problem and needs to be fixed.
> 
> Thanks for the analysis. That is exactly what is happening and what rpm
> should be preventing. Different builds should generate generate different
> build-ids. Which is what rpm tries to guarantee.
> 
> But the kernel package defeats rpm by calling debugedit directly and then
> overriding __debug_install_post and how find-debuginfo.sh is called.
> 
> The correct fix is for the kernel to call find-debuginfo.sh and debugedit
> like how rpm would. That means adding --ver-rel "%{VERSION}-%{RELEASE}" to
> the debuginfo_args and changing the debugedit call to include
> --build-id-seed=--ver-rel "%{VERSION}-%{RELEASE}". This is all a little
> involved because the kernel spec makes various assumptions about the precise
> workings of debugedit and find-debuginfo.sh and calls them (multiple times!)
> in different order.
> 
> So the easiest fix is reverting to the old setting for build-id links for
> the kernel by setting as follows:
> 
> diff --git a/kernel.spec b/kernel.spec
> index 9a5fe0b..9d7f7ba 100644
> --- a/kernel.spec
> +++ b/kernel.spec
> @@ -183,6 +183,9 @@ Summary: The Linux kernel
>  %define _enable_debug_packages 0
>  %endif
>  %define debuginfodir /usr/lib/debug
> +# Needed because we override almost everything involving build-ids
> +# and debuginfo generation. Currently we rely on the old alldebug setting.
> +%global _build_id_links alldebug
>  
>  # kernel PAE is only built on i686 and ARMv7.
>  %ifnarch i686 armv7hl
> 
> But if you do please don't close this bug yet. We should come up with a
> better way for the kernel package to do this that doesn't require so many
> hacks overriding rpm macros to generate correct buildids and debuginfo. It
> also prevents the kernel to take advantage of some pending upstream
> improvements to rpm for parallel debuginfo generation for example.

I don't really understand those command here. As an user, what should I do to get the rawhide updated?

(In reply to Bowen Wang from comment #10)
> (In reply to Mark Wielaard from comment #9)
> > So the easiest fix is reverting to the old setting for build-id links for
> > the kernel by setting as follows:
> > 
> > diff --git a/kernel.spec b/kernel.spec
> > index 9a5fe0b..9d7f7ba 100644
> > --- a/kernel.spec
> > +++ b/kernel.spec
> > @@ -183,6 +183,9 @@ Summary: The Linux kernel
> >  %define _enable_debug_packages 0
> >  %endif
> >  %define debuginfodir /usr/lib/debug
> > +# Needed because we override almost everything involving build-ids
> > +# and debuginfo generation. Currently we rely on the old alldebug setting.
> > +%global _build_id_links alldebug
> >  
> >  # kernel PAE is only built on i686 and ARMv7.
> >  %ifnarch i686 armv7hl
> > 
> > But if you do please don't close this bug yet. We should come up with a
> > better way for the kernel package to do this that doesn't require so many
> > hacks overriding rpm macros to generate correct buildids and debuginfo. It
> > also prevents the kernel to take advantage of some pending upstream
> > improvements to rpm for parallel debuginfo generation for example.
> 
> I don't really understand those command here. As an user, what should I do
> to get the rawhide updated?

You can either build your own kernel package with the above patch applied or wait till a kernel package maintainer has time to look at the report and apply the workaround if they think that is the best way forward. Then you'll get an updated kernel package without any of the build-id files that cause conflicts.

(In reply to Mark Wielaard from comment #11)
> (In reply to Bowen Wang from comment #10)
> > (In reply to Mark Wielaard from comment #9)
> > > So the easiest fix is reverting to the old setting for build-id links for
> > > the kernel by setting as follows:
> > > 
> > > diff --git a/kernel.spec b/kernel.spec
> > > index 9a5fe0b..9d7f7ba 100644
> > > --- a/kernel.spec
> > > +++ b/kernel.spec
> > > @@ -183,6 +183,9 @@ Summary: The Linux kernel
> > >  %define _enable_debug_packages 0
> > >  %endif
> > >  %define debuginfodir /usr/lib/debug
> > > +# Needed because we override almost everything involving build-ids
> > > +# and debuginfo generation. Currently we rely on the old alldebug setting.
> > > +%global _build_id_links alldebug
> > >  
> > >  # kernel PAE is only built on i686 and ARMv7.
> > >  %ifnarch i686 armv7hl
> > > 
> > > But if you do please don't close this bug yet. We should come up with a
> > > better way for the kernel package to do this that doesn't require so many
> > > hacks overriding rpm macros to generate correct buildids and debuginfo. It
> > > also prevents the kernel to take advantage of some pending upstream
> > > improvements to rpm for parallel debuginfo generation for example.
> > 
> > I don't really understand those command here. As an user, what should I do
> > to get the rawhide updated?
> 
> You can either build your own kernel package with the above patch applied or
> wait till a kernel package maintainer has time to look at the report and
> apply the workaround if they think that is the best way forward. Then you'll
> get an updated kernel package without any of the build-id files that cause
> conflicts.

Ok, thanks.

*** Bug 1433922 has been marked as a duplicate of this bug. ***

*** Bug 1433565 has been marked as a duplicate of this bug. ***

*** Bug 1433567 has been marked as a duplicate of this bug. ***

As mentioned on the mailing list, I applied the workaround to today's rawhide. Looking at debuginfo is on my TODO list this week.