Fedora Account System
Red Hat Associate
Red Hat Customer
Created attachment 376071 [details] Patch to update bsddb3 to 4.8.1 Description of problem: The current version of python in rawhide uses bsddb3 4.7.x. However, the Oracle DB version in rawhide has now been updated to 4.8, and python will fail to build if you try and build it against this. You can see that in this scratch build: Version-Release number of selected component (if applicable): python-2.6.4-3.fc13 How reproducible: Always Steps to Reproduce: 1. Build python in rawhide. 2. Watch it fail to build Actual results: Compile errors. Expected results: Working build Additional info: The attached patch updates the bsddb3 module to version 4.8.1 from: http://pypi.python.org/pypi/bsddb3/ You can see a successful scratch build of this here: http://koji.fedoraproject.org/koji/taskinfo?taskID=1849105
Created attachment 376072 [details] spec file patch
(In reply to comment #0) > Created an attachment (id=376071) [details] > Patch to update bsddb3 to 4.8.1 > > Description of problem: > > The current version of python in rawhide uses bsddb3 4.7.x. However, the > Oracle DB version in rawhide has now been updated to 4.8, and python will fail > to build if you try and build it against this. You can see that in this > scratch build: Oops, forgot to put URL to failing scratch build: http://koji.fedoraproject.org/koji/taskinfo?taskID=1849142
I'd be more than happy to commit this and build it if the change seems acceptable.
Sorry about belated response; I'm on the road (FUDCon) and have only very intermittent connectivity right now. Thanks for generating the patch. Do all the tests still pass if you run: python /usr/lib/python2.6/test/test_bsddb3.py (part of the python-test subpackage of our python rpms) I'm wondering how best to approach this, especially with respect to upstream. Am I right in thinking that you're attempting to bump the version of the dependent package in rawhide? What other packages needed work to deal with the API changes? Is there a good overview somewhere of these changes (do you have a feature page for this work?) I'm a little wary of carrying a big patch in our SRPM, but I'm not yet sure how best to solve this - if our aim is to keep support for the module within the "python" rpm, I wonder if would be better to simply ship the tarball from the standalone module within the python srpm, and use that code directly. (need to get back on the bus now)
Generally for new bdb versions, we've done significantly simpler patches. See, eg, F-10/python-2.5.2-db47.patch for the one that brought us up to working with bdb4.7 in python 2.5.2
Yeh, I mentioned to Josh that previous db bumps had been much smaller but I assumed that the patch was from upstream. He replied it was from upstream, and the reason it was so big was because the BDB API had changed (AIUI this only affects internal python, and not any python programs). David, I'm pretty sure just bundling the old version will have Spot opening a BZ to stop it :).
(In reply to comment #4) > Sorry about belated response; I'm on the road (FUDCon) and have only very > intermittent connectivity right now. > > Thanks for generating the patch. > > Do all the tests still pass if you run: > python /usr/lib/python2.6/test/test_bsddb3.py > (part of the python-test subpackage of our python rpms) I haven't tried. I don't have a rawhide machine available at the moment. > I'm wondering how best to approach this, especially with respect to upstream. > Am I right in thinking that you're attempting to bump the version of the > dependent package in rawhide? What other packages needed work to deal with the > API changes? Is there a good overview somewhere of these changes (do you have > a feature page for this work?) I didn't bump anything. The owner of the Oracle DB package bumped it in rawhide from 4.7 to 4.8. If you try to build the exising python against the new rawhide build root, it fails. You can see that with the scratch build URL in the previous comment. You'll hit this failure the next time you build python in rawhide, so I'm just trying to get a jump on it. > I'm a little wary of carrying a big patch in our SRPM, but I'm not yet sure how > best to solve this - if our aim is to keep support for the module within the > "python" rpm, I wonder if would be better to simply ship the tarball from the > standalone module within the python srpm, and use that code directly. Yeah, I understand the concerns about the patch. It is upstream, just in patch form. If you'd rather use a tarball, that's fine with me. You can see the overview of the changes for db4.8 here: http://www.jcea.es/programacion/pybsddb.htm Of particular note is the highlights for bsddb3-4.8.0
CCing jnovy (db4/compat-db maintainer) jnovy: how substantial are the API changes from 4.7 to 4.8? Here's a review of what we've done historically w.r.t. building python against a changing db4. FC-4, FC-5, FC-6: no patches; "BuildPrereq: db4-devel >= 4.3" from FC-5 onwards F-7 (Python-2.5)'s python.spec had: Patch6: python-db45.patch $ diffstat ../F-7/python-db45.patch Modules/_bsddb.c | 9 ++++++++- setup.py | 4 ++-- 2 files changed, 10 insertions(+), 3 deletions(-) F-8 and F-9 (both Python-2.5.1) had: Patch60: python-2.5.1-db46.patch $ diffstat ../F-8/python-2.5.1-db46.patch setup.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) Upstream merger for 4.6.4: http://svn.python.org/view?view=rev&revision=63207 F-10 (2.5.2) had: Patch60: python-2.5.2-db47.patch $ diffstat ../F-10/python-2.5.2-db47.patch Modules/_bsddb.c | 27 ++++++++++++++++++++++----- setup.py | 4 ++-- 2 files changed, 24 insertions(+), 7 deletions(-) F-11 (2.6) and F-12 (2.6.2) didn't have a patch for the bsddb code (or, rather, had the line from F-10 in commented-out form) Upstream support for 4.7 happened here: - merging from http://www.jcea.es/programacion/pybsddb.htm#bsddb3-4.7.3 in http://svn.python.org/view?view=rev&revision=66088 - http://www.jcea.es/programacion/pybsddb.htm#bsddb3-4.7.2 in http://svn.python.org/view?view=rev&revision=65193 - bsddb module updated to version 4.7.0 in http://svn.python.org/view?view=rev&revision=63537 Proposed patch is clearly much larger: $ diffstat python-2.6-update-bsddb3-4.8.patch Lib/bsddb/__init__.py | 14 Lib/bsddb/dbobj.py | 21 Lib/bsddb/dbtables.py | 9 Lib/bsddb/dbutils.py | 2 Lib/bsddb/test/test_all.py | 44 Lib/bsddb/test/test_associate.py | 66 - Lib/bsddb/test/test_basics.py | 163 ++ Lib/bsddb/test/test_compare.py | 22 Lib/bsddb/test/test_compat.py | 2 Lib/bsddb/test/test_dbenv.py | 148 ++ Lib/bsddb/test/test_dbtables.py | 16 Lib/bsddb/test/test_distributed_transactions.py | 6 Lib/bsddb/test/test_early_close.py | 13 Lib/bsddb/test/test_fileid.py | 63 + Lib/bsddb/test/test_lock.py | 15 Lib/bsddb/test/test_pickle.py | 12 Lib/bsddb/test/test_recno.py | 2 Lib/bsddb/test/test_replication.py | 25 Lib/bsddb/test/test_sequence.py | 8 Lib/bsddb/test_support.py | 54 Modules/_bsddb.c | 1426 ++++++++++++++++++++---- Modules/bsddb.h | 13 22 files changed, 1815 insertions(+), 329 deletions(-) Currently, python-2.6.4-3 in rawhide has an autogenerated requires on "libdb-4.7.so()" which is being satisfied by compat-db47. Running the (unpatched) python-2.6.4-3 code through /usr/lib64/python2.6/test/test_bsddb3.py shows 1 error: ====================================================================== FAIL: test01_basic_replication (bsddb.test.test_replication.DBReplicationManager) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.6/bsddb/test/test_replication.py", line 134, in test01_basic_replication self.assertTrue(time.time()<timeout) AssertionError ---------------------------------------------------------------------- Ran 340 tests in 43.275s FAILED (failures=1) I can see three approaches here: (a) apply the patch as is (b) ship the PyBSDDB tarball within our python SRPM, and use that directly, rather than carrying a patch between the Python-2.6.4.tar.bz2's version of the code vs this external tarball (c) try to come up with a smaller patch (d) build and link against compat-db47 instead Given that all of the prior patches came from the same source, I'm inclined to go with (a) for now. I needed to patch things a bit more to force it to link against 4.8; scratch built at http://koji.fedoraproject.org/koji/taskinfo?taskID=1878602 This scratch build gives full passes for the test suite on my x86_64 rawhide box: # python /usr/lib64/python2.6/test/test_bsddb3.py (snip) ---------------------------------------------------------------------- Ran 444 tests in 38.690s OK Tagged and building into dist-fc13 as python-2.6.4-4.fc13: http://koji.fedoraproject.org/koji/taskinfo?taskID=1878788
Just for the record, the db-4.8.24 release doesn't contain major compatibility-breaking changes. The benefit of upgrading is handful of fixes that 4.8.24 brings. If you are curious what changed in 4.8.24 from 4.7 BDB series, please have a look here: http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/changelog_4_8.html#id1655245 All changes are listed here: http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/changelog_4_8.html A good thing is that there is only one function added in case of the most widely used C API. There are only bugfixes in other APIs such as C++ and Java one.