So first let me say, desktop libraries in general don't support compiling against new version and running against an old version.  That's a level of ABI support they've never supported.  So for instance, there's probably similar issues with glib2 and gtk3 and various other libraries in the stack. So this may be WONTFIX territory ?

On the other hand, assuming we should fix this, i think the problem is caused by this commit:

https://cgit.freedesktop.org/dbus/dbus/commit/?id=57696a2e8a5eb2df3177f5b52c512a14f3cab1f5

I believe the point was to reduce the footprint of the dbus-daemon by having it link against libdbus instead of recompiling all the files from libdbus directly into the daemon binary.  Before it could link against libdbus, libdbus needed to start exporting private symbols.  But they need to be clearly demarcated as such, so applications don't try to use them.  

So, I guess, the idea was to introduce a linker version script:

--- /dev/null
+++ b/dbus/Version.in
@@ -0,0 +1,10 @@
+LIBDBUS_1_@SOVERSION@ {
+  global:
+    dbus_*;
+  local:
+    *;
+};
+LIBDBUS_PRIVATE_@DBUS_VERSION@ {
+  global:
+    _dbus_*;
+};

So you can see it's going to give all symbols with dbus_ as a prefix the symbol version LIBDBUS_1_3.  Given the original motivation for the version script, the only part of it that seems to matter is the LIBDBUS_PRIVATE_ part, not the LIBDBUS_1 part.  I think it could probably just be dropped? Another approach would be to assign a symbol version only to new symbols added since there was no versioning.  And, perhaps, change the version based on when a symbol was added.

CCing a couple of people that were in the discussion that motivated this change to see if they have anything to add.

> So first let me say, desktop libraries in general don't support compiling against new version and running against an old version. 

I don't think it's just desktop libraries (btw I wouldn't say libdbus is a "desktop library" nowadays if that's what you're implying).

That said I think what the reporter is getting at here (although it's not entirely clear) is that the dependency is purely synthetic.

But honestly this has been true of glibc for a long time...they add new versions of strcmp() or whatever that your app gets opted into by default.

While I think you're right in that originally upstream could have only assigned versions to new symbols, *dropping* them now would be a totally backwards incompatible change, right?

I think the bottom line on this bug is that it's probably a one-time hit.  libdbus isn't *likely* to add a new symbol with a version that existing applications get transparently linked to.  (This can happen though with e.g. static inlines or #defines in general)

So this seems like WONTFIX to me.

(In reply to Colin Walters from comment #4)
> I don't think it's just desktop libraries (btw I wouldn't say libdbus is a
> "desktop library" nowadays if that's what you're implying).
Well, i mean it's a library maintained by someone on desktop team.
But yea, most of the desktop doesn't even use it these days anyway. 

> While I think you're right in that originally upstream could have only
> assigned versions to new symbols, *dropping* them now would be a totally
> backwards incompatible change, right?
There could certainly be ways to handle this upstream that wouldn't be backward incompatible (aliases or whatever). It's slightly less complicated from a downstream point of view.  We never shipped the symbol versions in 7.4 so we could just drop them before shipping 7.5. would be pretty trivial, I think, but it would have to stick around as a downstream patch.

We could also go the compat- lib approach, but that seems a bit overkill for a "purely synthetic" incompatibility.

> So this seems like WONTFIX to me.
Okay, I'd like to get danpb's take, too, (and, of course, David King's) but WONTFIX doesn't seem wholly inappropriate to me.  Especially, if as you say, we already have the problem with glibc...

> We never shipped the symbol versions in 7.4 so we could just drop them before shipping 7.5. would be pretty trivial, I think, but it would have to stick around as a downstream patch.

Oh I missed that, I thought this was a post-release bug.  Yeah, in that case we *could* patch it downstream but...

I guess at this point let me ask - to what degree is this bug about "I had to check something in Errata Tool" vs "We suspect corosync users will find this problematic"?

yum is going to pull in the new libdbus for dependencies transparently right?  And corosync is part of base RHEL so there's no separate stream concerns I believe.

I admittedly did not look to binary expectations about the library,
but at least for me, the RPM-level change is plain bogus if/when nothing
changes about the binary compatibility.  That very RPM-level change
then means that trivial "if you don't want to move fully 7.4 -> 7.5
and you have an issue with corosync from 7.4, try updating just that
package" won't work as easily as in the past and will possibly result
in a noticeable amount of unnecessary updates, not just dbus itself,
but anything that was compiled with this extra-labelled version
(similar cases to mentioned corosync-2.4.3-1) just as well.

(Moreover, my vague memory tells me that mere dbus updating wasn't
always exactly pleasant experience on the live system in the past).

(In reply to Ray Strode [halfline] from comment #5)
> (In reply to Colin Walters from comment #4)
> > I don't think it's just desktop libraries (btw I wouldn't say libdbus is a
> > "desktop library" nowadays if that's what you're implying).
> Well, i mean it's a library maintained by someone on desktop team.
> But yea, most of the desktop doesn't even use it these days anyway. 
> 
> > While I think you're right in that originally upstream could have only
> > assigned versions to new symbols, *dropping* them now would be a totally
> > backwards incompatible change, right?
> There could certainly be ways to handle this upstream that wouldn't be
> backward incompatible (aliases or whatever). It's slightly less complicated
> from a downstream point of view.  We never shipped the symbol versions in
> 7.4 so we could just drop them before shipping 7.5. would be pretty trivial,
> I think, but it would have to stick around as a downstream patch.
> 
> We could also go the compat- lib approach, but that seems a bit overkill for
> a "purely synthetic" incompatibility.
> 
> > So this seems like WONTFIX to me.
> Okay, I'd like to get danpb's take, too, (and, of course, David King's) but
> WONTFIX doesn't seem wholly inappropriate to me.  Especially, if as you say,
> we already have the problem with glibc...

I consider this NOTABUG / WONTFIX. RHEL has never promised the ability to arbitrarily mix & match builds across different major versions. If you build against version N, you cannot expect that you will be able to run or install with N-1.  The C API exported is not the only thing that influences the ABI of a build, toolchain build options are similar. It is not uncommon for glibc to introduce new variants of a pre-existing function with versioned symbols, in which case you cannot use older glibc if you built with newer glibc.

Furthermore, it is not possible to remove symbol versioning, as that would be an ABI break for anything built against the current versioned library.

[comment 10] was meant for another bug, sorry.

Anyway,

> Furthermore, it is not possible to remove symbol versioning, as that
> would be an ABI break for anything built against the current
> versioned library.

Were such packages released officialy?  If not, the affected packages
can always be rebuilt, hopefully RCM can automate that.

Jan, first let me say, thanks for the bug report and for putting customers first.    It's good that this issue got discussion before the release.

You're right that the problematic builds haven't been released yet, so we could definitely go through the rigmarole to fix it.  On the other hand that rigmarole itself is a clear downside, with some risk.  Also, fixing this would require having to carry a patch we can't upstream. 

The upside is it makes doing something unsupported by the distribution easier.  

I think at this point we've got enough people in the WONTFIX camp to make WONTFIX the most natural action, so I'm going to close this out. 

Of course, if David King feels strongly otherwise, then we can reopen.

I won't oppose that if this is the last say on the topic in this very
instance.

However, this is apparently a pain point in our procedures, because
formalist changes like this will technnically serve no purpose and will
just artificially impose nonsensical restrictions, furthermore amplified
by viral spreading onto new packages depending on particular source
as they get built.

Ad absurdum, if the goal was to make whole set of packages assuredly
update all-at-once (unless circumvented), there could be an artifical
provides/requires injector for the existing packages, either targetting
virtual (similar to the automatism around libraries) or real (minimum
version for dependency foo).  Apparently, that's a nonsense just as well,
otherwise it would be in use already.

I can imagine (and I think I could back it with the hard data), that
sometimes the immediate workaround for some issues in otherwise
locked-on-particular-distro-release deployments is to pick the newer
package from the newer release, which usually works given the stability
guarantees in base OS.  This time, it won't work for corosync in this
particular case (whole a lot of other packages will need to be updated),
but we should make the process less welcoming to these disruptions,
perhaps requiring sign-off by some release engineer.

(In reply to Jan Pokorný from comment #13)
> I won't oppose that if this is the last say on the topic in this very
> instance.
> 
> However, this is apparently a pain point in our procedures, because
> formalist changes like this will technnically serve no purpose and will
> just artificially impose nonsensical restrictions, furthermore amplified
> by viral spreading onto new packages depending on particular source
> as they get built.

It is not quite that strict.  When symbol versioning is first added to a library, existing exported symbols will all get annotated with the chosen version. In this case all existing dbus symbols get LIBDBUS_1_3 as their elf version.  So packages built against this version can't be run with a version prior to symbol versioning being turned on.

Going forward though, things are more relaxed.  In libdbus 1.4, all the pre-existing symbols will still have LIBDBUS_1_3 as their version.  Only new APIs added in libdbus 1.4 would have  LIBDBUS_1_4 symbol version.

So if a downstream application builds against libdbus 1.4 and does *not* use any of the newly added APIs, it will be able to run against either libdbus 1.3 or libdbus 1.4.

This is a good thing, because it means that going forward the RPM dependencies against libdbus will encode the right minimum version of libdbus needed to satisfy the particular set of APIs that the downstream app needs. 

IOW, if an application does use a new API from libdbus 1.4, the automatic RPM dep will automatically ensure that the application cannot be installed with libdbus 1.3. This is a more fine grained and accurate dependency system that you get with non-versioned symbols. We just suffer the one time hit in the particular release that first introduces symbol versioning to a previously non-versioned library.  This is a net win overall in the long term.

(In reply to Daniel Berrange from comment #14)
> Going forward though, things are more relaxed.  In libdbus 1.4, all the
> pre-existing symbols will still have LIBDBUS_1_3 as their version.  Only new
> APIs added in libdbus 1.4 would have  LIBDBUS_1_4 symbol version.
Is that true as currently implemented?  Note the version script in comment 3. If the so name gets bumped, all symbols get the new version afaict.

The script just does dbus_* as a glob.

(In reply to Ray Strode [halfline] from comment #15)
> (In reply to Daniel Berrange from comment #14)
> > Going forward though, things are more relaxed.  In libdbus 1.4, all the
> > pre-existing symbols will still have LIBDBUS_1_3 as their version.  Only new
> > APIs added in libdbus 1.4 would have  LIBDBUS_1_4 symbol version.
> Is that true as currently implemented?  Note the version script in comment
> 3. If the so name gets bumped, all symbols get the new version afaict.
> 
> The script just does dbus_* as a glob.

Oh well if you actually change the soname, you've broken ABI compat regardless, so it doesn't matter if you reset symbol versions at that point. What I wrote above was just assuming that soname is not being changed between point releases in general.

okay well configure.ac has this in it:

SOVERSION=`expr ${LT_CURRENT} - ${LT_AGE}`

both LT_CURRENT and LT_AGE are going to get incremented in lockstep if a new interface is added, so SOVERSION should remain 3 indefinitely I think.

I do agree that what you propose, incrementing the symbol version when adding new APIs would make more sense.

Yeah, that is rather missing key benefit of using versioned symbols and well worth changing IMHO.

I chatted with upstream about this a bit (slightly edited for clarity):

<halfline> smcv: thoughts on https://bugzilla.redhat.com/show_bug.cgi?id=1536352 ?
<smcv> halfline: I think WONTFIX is the correct answer to https://bugzilla.redhat.com/show_bug.cgi?id=1536352 , if you arbitrarily upgrade libdbus from one stable branch to another, newly compiled binaries are free to depend on that newer version
<bugbot> Bug 1536352: Red Hat Enterprise Linux 7, urgent, unspecified, rc, dking, CLOSED WONTFIX, Automatic Provides added libdbus-1.so.3(LIBDBUS_1_3)(64bit) to libdbus-1.so.3()(64bit) for dbus-libs, meaning the packages compiled against newer dbus cannot be installed together with older dbus despite allegedly no ABI break occurs
<halfline> smcv: what about the side point that maybe newer symbols should get newer symbol versions ?
<smcv> halfline: Daniel Berrange is right to say that we are only getting half of the benefit of versioned symbols. it's a trade-off - libdbus doesn't add ABI frequently enough that it seemed worth my time to maintain full-fat symbol versioning
<smcv> halfline: so I'm doing the libssl/libpng-style "use version as super-SONAME" not the e.g. telepathy-glib-style "annotate every symbol with the version that introduced it"
<smcv> this means if we *do* one day break ABI, anything that has been recompiled since we added the symbol versioning is immune to the "pull in libdbus-1.so.3 via one dependency, pull in libdbus-1.so.4 via another, boom" sort of bug
<halfline> ah, that's true, interesting.
<smcv> the Debian release team are very very keen on library maintainers doing that even if they can't justify the effort to do the full abi versioning thing like telepathy-glib or libostree
<smcv> if I had done the full ABI versioning thing then I suspect it would have taken way longer to get right and get reviewed, so it was a bit of a perfect is the enemy of the good situation
<halfline> i guess ideally the first commit would have added LIBDBUS_PRIVATE_ for _dbus_* to address slimming down dbus-daemon; and then a follow up commit would have added LIBDBUS_1_ for dbus_* and the second commit message could have mentioned thie furture proofing against two libdbus's getting loaded at once
<halfline> anyway doesn't really matter at this point of course
<smcv> halfline: yeah I wasn't sure how portable or safe or reliable it was to be versioning some-but-not-all symbols
<smcv> I'm sure it works, but aiui approximately nobody does that
<halfline> yea i think it works fine from my reading of the info page anyway
<halfline> thanks for the explanation, very helpful !
<halfline> okay, good enough for me
* halfline closes tab and moves on :-)