Bug 1905584 - tracker: postgresql and libpq packaging design change [NEEDINFO]
Summary: tracker: postgresql and libpq packaging design change
Alias: None
Product: Fedora
Classification: Fedora
Component: postgresql
Version: 34
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Honza Horak
QA Contact: Fedora Extras Quality Assurance
Depends On: 1926592 1926593 1926594
TreeView+ depends on / blocked
Reported: 2020-12-08 15:40 UTC by Patrik Novotný
Modified: 2021-08-23 12:36 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2021-08-23 12:36:30 UTC
Type: Bug
hhorak: needinfo? (panovotn)

Attachments (Terms of Use)

Description Patrik Novotný 2020-12-08 15:40:25 UTC
As of now, we are building the libpq package separately from the postgresql package. Both packages build from the same source repository.

By separating libpq from the postgresql package, it is possible for modular postgresql builds to share a single non-modular libpq package as a dependency.

Until the upstream release 12.2, this approach was working beautifully. With this release, however, we've learned that we can't reliably build updated modular postgresql packages against the libpq package, without it being updated as well.

For example, we couldn't build postgresql:10 updated to version 10.13 against libpq in version 12.1 (released at that time), despite version 12.1 being higher then 10.13.

That is because the version 10.13 has been released two release cycles after the version 12.1. Actually, version 12.3 has been released in the same cycle as the version 10.13.

As PostgreSQL upstream often backports features across their major versions, in this case, there were features introduced in 12.3, those features were backported to 10.13, but still missing in our release of libpq in version 12.1. Thus, to update the postgresql package to 10.13, the libpq package had to be updated to version 12.3.

The above already means we're not benefiting from the split of libpq as much as anticipated. Though, more concerning was the upstream release of version 13.1.

Since some features introduced[1] in libpq 13.1 weren't backported to any other major version supported by upstream, and caused a minor incompatibility for postgresql:12, postgresql:10 and postgresql:9.6, while postgresql:13 relies on them, we need to resolve this incompatibility downstream.

That is not ideal. Especially as there could be more serious incompatibilities in future releases, and/or they could be hard to detect. Also, upstream has advised strongly against such packaging design[2].

To address this issue, while maintaining the separate libpq build, which is beneficial in certain use-cases, we will introduce a new packaging design:

  - each postgresql modular build will bundle corresponding version of the libpq library
  - this bundled libpq will be hidden by prefixing the library SONAME
  - the non-modular libpq package will be kept as is, and maintained independently from the postgresql packages

This change will be included in a planned F34 change proposal, which I will reference here once completed.

[1] https://www.postgresql.org/message-id/ce7881f6-bf37-7696-0f1f-e7f07eaaaaf5@2ndQuadrant.com
[2] https://www.postgresql.org/message-id/2147994.1592920488%40sss.pgh.pa.us

Comment 1 Patrik Novotný 2021-01-13 13:50:55 UTC
Update to 1.5.0 for PostgreSQL 13 support


Comment 2 Patrik Novotný 2021-01-13 14:29:06 UTC
Update postgresql to 13.1 and postgresql-setup to v8.5


Comment 3 Honza Horak 2021-01-13 15:41:45 UTC
Update of postgres-decodebufs: https://src.fedoraproject.org/rpms/postgres-decoderbufs/pull-request/4

Comment 4 Patrik Novotný 2021-01-13 20:27:51 UTC
Update for rhdb-utils: https://src.fedoraproject.org/rpms/rhdb-utils/pull-request/1

Comment 5 Patrik Novotný 2021-01-13 23:00:15 UTC
Update for postgresql-pgpool-II: https://src.fedoraproject.org/rpms/postgresql-pgpool-II/pull-request/6

Comment 6 Patrik Novotný 2021-01-14 08:46:08 UTC
Update for pgsphere: https://src.fedoraproject.org/rpms/pgsphere/pull-request/2

Comment 7 Patrik Novotný 2021-01-14 09:43:07 UTC
Update for pg_repack: https://src.fedoraproject.org/rpms/pg_repack/pull-request/1

Comment 8 Patrik Novotný 2021-01-15 10:00:11 UTC
Update for orafce: https://src.fedoraproject.org/rpms/orafce/pull-request/2

Comment 9 Patrik Novotný 2021-01-15 10:30:24 UTC
Update for postgresql-ip4r: https://src.fedoraproject.org/rpms/postgresql-ip4r/pull-request/1

Comment 10 Honza Horak 2021-02-05 16:31:42 UTC
This is a PoC of a change to build the postgresql server with own copy of libpq (with mangled soname to not conflict with the libpq itself):

Comment 12 Ben Cotton 2021-02-09 15:32:39 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 13 Honza Horak 2021-06-04 16:02:28 UTC
The current solution for building postgresql with a private libpq does not address the case when a server extension links to libpq as well. Currently, the postgresql-server-devel does not ship enough to allow such an extension to build without libpq installed, and with libpq installed we can get to situation when the system libpq and also the private libpq from the server are linked together in a single process. This needs some improvement.

Comment 14 Honza Horak 2021-06-08 09:13:31 UTC
(In reply to Honza Horak from comment #13)
> The current solution for building postgresql with a private libpq does not
> address the case when a server extension links to libpq as well.

This PR should address this case as well:

In short, the private libpq is now available in a sub-package private-libs and private-devel, including the unversioned .so library and the header files, so extensions that need to link to libpq as well (like pg_repack), can do so with the private libpq.

Comment 15 Honza Horak 2021-06-09 16:20:38 UTC
(In reply to Honza Horak from comment #14)
> (In reply to Honza Horak from comment #13)
> > The current solution for building postgresql with a private libpq does not
> > address the case when a server extension links to libpq as well.
> This PR should address this case as well:
> https://src.fedoraproject.org/rpms/postgresql/pull-request/32

This change also adds an RPM requirement from postgresql-server-devel to either libpq-devel or private-devel, depending on whether server uses external libpq or not. With this, extensions like pg_repack do not need to have a dependency on libpq any more and can leave it up to postgresql-server-devel package to pull in some libpq (every-time).

Comment 16 Honza Horak 2021-06-09 16:22:12 UTC
@panovotn@redhat.com @fjanus@redhat.com I'd be happy if you can take a look at the PR in comment #14 and say what do you think about such approach.

Note You need to log in before you can comment on or make changes to this bug.