Bug 1779267 - Remove libdb dependency from apr-util
Summary: Remove libdb dependency from apr-util
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: apr-util
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Luboš Uhliarik
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1778802 1940391
TreeView+ depends on / blocked
 
Reported: 2019-12-03 15:55 UTC by Filip Januš
Modified: 2023-12-12 14:22 UTC (History)
4 users (show)

Fixed In Version: apr-util-1.6.3-13.fc40
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1940391 (view as bug list)
Environment:
Last Closed: 2023-12-12 14:22:58 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github apache apr pull 46 0 None closed WIP: lmdb support 2023-10-10 14:21:25 UTC
Github apache apr pull 49 0 None open apr_dbm_lmdb.c: better error handling and better 2023-10-10 14:21:25 UTC

Description Filip Januš 2019-12-03 15:55:12 UTC
Description of problem:
According to more restrictive libdb licence policy exists effort to remove libdb's dependencies.
apr-util package is now built with libdb requirement, this package could be build without libdb (configure --without-berkeley-db). libdb coud be replaced by gdbm or SQLite(configure --with-sqlite3=DIR).




Actual results:
apr-util requires libdb package

Expected results:
run without libdb

Comment 1 Joe Orton 2019-12-05 16:30:20 UTC
apr-util's DBM interface (apr_dbm.h) can use a variety of backends and currently defaults to Berkeley DB, although SDBM is also supported.

Switching the default from BDB to SDBM - and dropping BDB support, is possible, though it might be disruptive for consumers of the DBM interface.

Do you have a target Fedora release for which this is intended?  Possibly we could look at supporting e.g. LMDB through this interface as a better alternative to SDBM.

Comment 2 Filip Januš 2019-12-09 12:25:06 UTC
In the best case it should be Fedora 32 or at least Fedora 33.

Comment 3 Ben Cotton 2020-02-11 17:41:38 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 4 Filip Januš 2020-11-23 13:48:56 UTC
We would like to move a step forward by getting rid of libdb. So I would like to ask, which problems do you see with changing backend from BDB to some other database? I want to summarize as much as possible problems with this change across components, so we can find the best possible solution.

Comment 5 Joe Orton 2021-03-18 10:59:09 UTC
Lubos is working on this, seeing if we can switch to LMDB.  This one is in progress.

Comment 6 Fedora Program Management 2021-04-29 16:00:44 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 7 Ben Cotton 2021-05-25 15:12:58 UTC
Fedora 32 changed to end-of-life (EOL) status on 2021-05-25. Fedora 32 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 8 Ben Cotton 2022-02-08 21:38:59 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 9 Filip Januš 2023-01-27 10:30:42 UTC
Is there any progress? May I help somehow?

Comment 10 Ben Cotton 2023-02-07 14:51:23 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 38 development cycle.
Changing version to 38.

Comment 12 Joe Orton 2023-10-10 14:19:48 UTC
The LMDB driver is getting merged upstream now. We'll backport and switch the default from libdb in time for F40.

Comment 13 Fedora Update System 2023-10-26 20:58:03 UTC
FEDORA-2023-f74acc9cfb has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-f74acc9cfb

Comment 14 Fedora Update System 2023-10-27 02:15:23 UTC
FEDORA-2023-f74acc9cfb has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-f74acc9cfb`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-f74acc9cfb

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 15 Joe Orton 2023-11-02 14:30:55 UTC
The git tests have some WebDAV mode and are breaking because of this.

https://src.fedoraproject.org/rpms/git/pull-request/20#comment-0

(also the deps broke but that was trivial to fix)

from some debugging it looks like mdb_txn_begin() calls are intermittently failing with EINVAL

Comment 16 Joe Orton 2023-11-02 14:44:50 UTC
This looks like:

https://bugs.openldap.org/show_bug.cgi?id=9486#c1 

the statement from Howard Chu is a bit worrying, that it's not safe
to try opening an LMDB database from multiple processes concurrently?!

Comment 17 Joe Orton 2023-11-02 16:50:58 UTC
I tried adding a sleep-and-retry loop around the mdb_txn_begin() call, it possibly helped a bit but even with significant delays it can still fail.

I reverted the default to building BDB support again to fix the git tests.

Backup plans:

a) use an external lock file around the MDB file managed by the apr_dbm layer rather than relying on LMDB doing locking/exclusion.  Can base this off the existing SDBM locking code, simple and foolproof

b) give up and use SDBM as the default DBM

Comment 18 Joe Orton 2023-12-12 09:47:16 UTC
httpd-2.4.58-3.fc40 adds locking in mod_dav_fs which reliably fixes the locking issues in my testing.

So it should be safe to switch back to LMDB by default now.

Comment 20 Joe Orton 2023-12-12 09:56:54 UTC
Package: apr-util-1.6.3-13.fc40
Build: https://koji.fedoraproject.org/koji/buildinfo?buildID=2332591

Comment 21 Joe Orton 2023-12-12 14:22:58 UTC
Landed in Rawhide


Note You need to log in before you can comment on or make changes to this bug.