It was discovered, that OpenOffice.org memory allocator is not 64bit clean.
rtl_allocateMemory() function in sal/rtl/source/alloc_global.c accepts one
argument - sal_Size n. On 64bit platforms such as x86_64, sal_Size is defined
as unsigned long int. This requested memory chunk size is later memory aligned
as size (type sal_Size). size is later used to calculate int index to
int index = (size - 1) >> RTL_MEMALIGN_SHIFT;
However, as sizeof(int) == 4 and sizeof(sal_Size) == 8 on 64bit platforms, the
calculated value may not fit into index (this can happen when
rtl_allocateMemory() is called with large argument, e.g. when some other flaw
causes OpenOffice to attempt to allocate chunk of memory with negative size,
which wraps to large positive value during signed -> unsigned type conversion,
such as ). Value stored in the index is wrapped / truncated, possibly
resulting in the index being negative.
Before index is used, it is checked not to exceed fixed upper limit, but it's
not checked whether its value is >= 0:
if (index < RTL_MEMORY_CACHED_LIMIT >> RTL_MEMALIGN_SHIFT)
Negative index used in g_alloc_table[index] will cause OpenOffice to access
memory outside of the g_alloc_table array. This may result it crash, or if
that points to an attacker controlled memory, attacker may possibly be able to
use this flaw to run arbitrary code.
OpenOffice.org upstream is aware of this issue, but they do not plan to release any security advisory for it, as they do not ship any 64bit pre-built packages. Issue was fixed upstream shortly after we reported it. Upstream bug report:
This issue does not affect openoffice.org and openoffice.org2 packages as shipped in Red Hat Enterprise Linux 3 and 4, as only 32bit builds are shipped in those versions.
Rene Engelhard of Debian pointed out that 64bit builds that use system memory allocator instead of the OpenOffice.org custom one (i.e. compiled with --with-alloc=system) are also unaffected by this problem.
*** Bug 455867 has been marked as a duplicate of this bug. ***
openoffice.org-2.4.1-17.6.fc9 has been pushed to the Fedora 9 stable repository. If problems still persist, please make note of it in this bug report.
openoffice.org-2.3.0-6.16.fc8 has been pushed to the Fedora 8 stable repository. If problems still persist, please make note of it in this bug report.
This issue was addressed in:
Red Hat Enterprise Linux: