Bug 1695525 - Fails to map large DBs on i686/s390x
Summary: Fails to map large DBs on i686/s390x
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: qemu
Version: 31
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Fedora Virtualization Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-03 09:40 UTC by Elliott Sales de Andrade
Modified: 2020-11-24 18:54 UTC (History)
11 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2020-11-24 18:54:30 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Elliott Sales de Andrade 2019-04-03 09:40:25 UTC
Description of problem:
The documentation for LMDB [1] states that the map size should be "chosen as large as possible". Consequently, the zarr developers have chosen a map size of 2**40 on 64-bit systems and 2**28 on 32-bit systems [2]. When the package is built on a 64-bit system [3], everything works fine. However, if the build is run on a 32-bit system [4], it fails with a MemoryError.

I have narrowed this down to the mmap call at [5] which attempts to map the backing file into memory. AFAICT, the mapped size is far below the RAM on the build machine as well as far below the normal 32-bit VM limit. So I don't know why the call is failing.

I can reproduce this in mock with the fedora-rawhide-i386 config, but what's weird is it also fails in the fedora-rawhide-s390x config. This machine has 16G of RAM, so it's enough for 2**28, but not 2**40. So it fails on 32-bit x86 with more than enough RAM, works on 64-bit x86 with nowhere near enough RAM, yet fails with 64-bit s390x.


Version-Release number of selected component (if applicable):
0.9.23-2.fc30


Steps to Reproduce:
1. mock -r fedora-rawhide-x86_64 ./python-zarr-2.3.1-2.fc31.src.rpm (works)
2. mock -r fedora-rawhide-i386 ./python-zarr-2.3.1-2.fc31.src.rpm (fails)
3. mock -r fedora-rawhide-s390x --forcearch=s390x ./python-zarr-2.3.1-2.fc31.src.rpm (fails)

Just to keep zarr out of it though, you can also do something like:
$ mock -r fedora-rawhide-s390x --forcearch=s390x --install python3-lmdb
$ mock -r fedora-rawhide-s390x --forcearch=s390x --shell
# python3
>>> import lmdb
>>> lmdb.open("test.db", map_size=2**40)


Actual results:
MemoryError in 4-6 tests


Expected results:
Tests all pass.


Additional info:
[1] http://www.lmdb.tech/doc/group__mdb.html#gaa2506ec8dab3d969b0e609cd82e619e5
[2] https://github.com/zarr-developers/zarr/blob/f6ced1e31b919065f8834d813ec081d2a85195b3/zarr/storage.py#L1585-L1587
[3] https://koji.fedoraproject.org/koji/taskinfo?taskID=33910305
[4] https://koji.fedoraproject.org/koji/taskinfo?taskID=33881973
[5] http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=libraries/liblmdb/mdb.c;h=e12af4482a0172da8d759c1da1530339d9095510;hb=2a5eaad6919ce6941dec4f0d5cce370707a00ba7#l4019

Comment 1 Jan Staněk 2019-04-16 08:40:35 UTC
I have trouble setting up the s390x mock, so I have tested with i686 mock, and I cannot reproduce it so far:

$ mock -r fedora-rawhide-i386 --forcearch=i686 --shell
# python3
>>> import lmdb
>>> lmdb.open("/tmp/test.db", map_size=2**40)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Integer argument exceeds limit.  # expected
>>> lmdb.open("/tmp/test.db", map_size=2**28)
<Environment object at 0xf7631448>              # no error
>>> _.close()

As a side note, this seems to be rather low-level issue, and I do not have much of an idea on what might be wrong.
I suggest asking the authors of lmdb directly (http://www.openldap.org/lists/) as they might have a better understanding of the issue. Alternatively, try the devel.org to reach a wider audience – perhaps someone will have an idea.

Comment 2 Dan Horák 2019-06-04 09:27:35 UTC
I don't see a problem when building in rawhide on s390x, within a 4GB VM ...

Wrote: /home/sharkcz/python-zarr/python-zarr-2.3.2-1.fc31.src.rpm

Comment 3 Elliott Sales de Andrade 2019-06-06 08:25:41 UTC
Thanks to Tom's message on the mailing list, I figured out that tests don't explicitly close backing stores leaving it to gc. This leaves several 2**28 mmaps around so that it eventually runs out of virtual memory space. I added a patch to close things and it now builds on all arches in koji.

This still fails in qemu'd mock:

$ mock -r fedora-rawhide-s390x --forcearch=s390x --install python3-lmdb
$ mock -r fedora-rawhide-s390x --forcearch=s390x --shell
# python3
>>> import lmdb
>>> lmdb.open("test.db", map_size=2**40)

but I'm less concerned about that now that I can get koji working properly.

I don't know if it's a qemu issue or elsewhere, but if you wish to close, that is fine by me (though figuring out the qemu mock issue might be nice.)

Comment 4 Dan Horák 2019-06-06 08:37:32 UTC
I suppose this is a qemu emulation issue.

Comment 5 Ben Cotton 2019-08-13 16:56:44 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 6 Ben Cotton 2019-08-13 19:15:07 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 7 Elliott Sales de Andrade 2019-10-21 20:13:24 UTC
Re-assigning to qemu, as noted in comment 4.

Comment 8 Dan Horák 2019-10-24 10:09:32 UTC
wonder if it could be related to bug #1741935, but sure it makes sense to retry with latest qemu

Comment 9 Ben Cotton 2020-11-03 15:12:52 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Ben Cotton 2020-11-24 18:54:30 UTC
Fedora 31 changed to end-of-life (EOL) status on 2020-11-24. Fedora 31 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.