Bug 1116406 - botan overwrites gmp's default memory functions
Summary: botan overwrites gmp's default memory functions
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: botan
Version: 20
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Thomas Moschny
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1123803
TreeView+ depends on / blocked
 
Reported: 2014-07-04 13:20 UTC by Nikos Mavrogiannopoulos
Modified: 2015-03-15 00:58 UTC (History)
1 user (show)

Fixed In Version: botan-1.10.9-4.fc20
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-09 09:47:26 UTC


Attachments (Terms of Use)

Description Nikos Mavrogiannopoulos 2014-07-04 13:20:45 UTC
Description of problem:
In GMP_Engine::GMP_Engine() botan does call:
      mp_set_memory_functions(gmp_malloc, gmp_realloc, gmp_free);

That overwrites the global gmp memory allocation functions. On the destructor it sets them to NULL. That is very very bad when one would dynamically load a library using botan.

I realized that if I load softhsm (which uses botan) from gnutls as a PKCS #11 module, and at some point unload it, I'll have a crash on gnutls which also uses gmp. That is because for the duration that the module is loaded gnutls is using botan's allocation functions, and when the module is unloaded structures are released using the default allocation functions.

As a rule libraries shouldn't change global process allocation functions, and they shouldn't assume they are the only user of gmp.

Comment 1 Thomas Moschny 2014-08-09 07:03:42 UTC
Talked to upstream.

Besides the obvious 'well don't unload it then' :) we have to options:
- disabling the gmp engine
- just disabling the allocator override

Opinions?

Comment 2 Nikos Mavrogiannopoulos 2014-08-09 09:20:53 UTC
- Don't unload isn't an option because softhsm is a module intended to be loaded and unloaded at runtime. Otherwise botan might be a bad choice for softhsm.

- Disabling the gmp engine. I have no idea what it is. Do you mean compiling botan without gmp? That's up to you as a maintainer. If it solves the issue, I'm ok with it.

- Just disabling the allocator override: I believe that's the most reasonable option.

Comment 3 Nikos Mavrogiannopoulos 2014-08-09 09:22:39 UTC
(In reply to Nikos Mavrogiannopoulos from comment #2)
> - Don't unload isn't an option because softhsm is a module intended to be
> loaded and unloaded at runtime. Otherwise botan might be a bad choice for
> softhsm.

Moreover it doesn't solve anything in the scenario I described above. By the time botan is loaded with softhsm there may be already structures allocated by another user of gmp (gnutls in that case).

Comment 4 Thomas Moschny 2014-08-10 21:41:23 UTC
So imho that's a deficit of gmp that one can only override its allocation functions globally.

Anyway, I think I will patch Botan and disable the override.

Comment 5 Fedora Update System 2015-02-26 19:08:46 UTC
botan-1.10.9-4.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/botan-1.10.9-4.el7

Comment 6 Fedora Update System 2015-02-26 19:08:55 UTC
botan-1.10.9-4.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/botan-1.10.9-4.fc20

Comment 7 Fedora Update System 2015-02-26 19:08:59 UTC
botan-1.10.9-4.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/botan-1.10.9-4.fc21

Comment 8 Fedora Update System 2015-03-09 08:29:42 UTC
botan-1.10.9-4.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2015-03-09 08:32:33 UTC
botan-1.10.9-4.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2015-03-15 00:58:55 UTC
botan-1.10.9-4.el7 has been pushed to the Fedora EPEL 7 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.