Bug 1405047 - Request to update arm64 jemalloc
Summary: Request to update arm64 jemalloc
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: jemalloc
Version: 25
Hardware: aarch64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Ingvar Hagelund
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: ARMTracker
TreeView+ depends on / blocked
 
Reported: 2016-12-15 12:56 UTC by cmuellner
Modified: 2017-01-30 15:05 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-30 15:05:46 UTC


Attachments (Terms of Use)

Description cmuellner 2016-12-15 12:56:28 UTC
I am using Fedora 25 an arm64/aarch64 machine.

With the jemalloc version of arm64 (4.2.1), I see a lot of warnings of the form
"munmap(): Invalid parameter" when running HHVM.
This does not happen with other arm64 distributions (e.g. Ubuntu Xenial
or OpenSuse LEAP 42.2), but these also use different versions of jemalloc.

I've been able to successfully get rid of the problem by uninstalling
jemalloc and compiling jemalloc-4.3.1.tar.bz2 on my own with
./configure && make && sudo make install

I've seen that jemalloc is already at 4.3.1 on i686, x86_64 and armv7hl (see [1]),
while arm64 still uses 4.2.1 (see [2]).

Therefore I'd like to ask to trigger a build of jemalloc 4.3.1 for arm64.

Thanks,
Christoph

[1] https://koji.fedoraproject.org/koji/packageinfo?packageID=11238
[2] https://arm.koji.fedoraproject.org/koji/packageinfo?packageID=10812

Comment 1 Ingvar Hagelund 2016-12-16 07:35:00 UTC
Hey, Christoph

While waiting for the arm-guys to bump the package, you may try this build:

https://arm.koji.fedoraproject.org/koji/taskinfo?taskID=3786788

Peter, do you know why the package hasn't been bumped for aarch64?

Ingvar

Comment 2 cmuellner 2016-12-16 22:31:23 UTC
Hi Ingvar,

I just installed your RPMs and tested with them.
Works perfectly on my arm64 machine. :)

Thanks a lot,
Christoph

Comment 3 Peter Robinson 2016-12-20 07:13:09 UTC
> Peter, do you know why the package hasn't been bumped for aarch64?

Because you didn't submit it as an update on primary (note the f25-updates-candidate tag) hence it won't mirror over until it's in testing.

https://koji.fedoraproject.org/koji/buildinfo?buildID=816783

And 4.4 is FTBFS in rawhide too btw https://koji.fedoraproject.org/koji/buildinfo?buildID=823900

Comment 4 Ingvar Hagelund 2016-12-21 10:20:33 UTC
> And 4.4 is FTBFS in rawhide too btw
> https://koji.fedoraproject.org/koji/buildinfo?buildID=823900

Yes, one of the tests fails on all but x86*. It's probably just a porting bug in the test, like size_t or something. I'm waiting for upstream. https://github.com/jemalloc/jemalloc/issues/526

Comment 5 cmuellner 2017-01-27 16:39:09 UTC
Hi Ingvar,

this week I ran "dnf update" on my aarch64 server and jemalloc got updated to 4.4.0.
Thanks for that!

However, my original problem (I see a lot of warnings of the form
"munmap(): Invalid parameter" when running HHVM) still persist.

I was able to solve this problem by rebuilding using the following commands:

dnf download --source jemalloc
rpmbuild --rebuild jemalloc-4.4.0-2.fc25.src.rpm
sudo dnf reinstall ~/rpmbuild/RPMS/aarch64/{jemalloc-4.4.0-2.fc25.aarch64.rpm,jemalloc-debuginfo-4.4.0-2.fc25.aarch64.rpm,jemalloc-devel-4.4.0-2.fc25.aarch64.rpm}

Build log can be found here:
https://paste.fedoraproject.org/537774/

I compared the build log with the Fedora 25 build log from here:
https://arm.koji.fedoraproject.org/kojifiles/packages/jemalloc/4.4.0/2.fc25/data/logs/aarch64/build.log

The differences are as follows:

 * In my build log there is no "Patch #5 (jemalloc-4.4.0.disable_thp.patch)" patching
 * My pagesize is 4096 (Fedora build machine has 65536)
 * My kernel is 4.3 based (Fedora 25 build server uses 4.7)
 * My kernel supports transparent_hugepage as well
 * My jemalloc configure finds autoconf
 * My LG_PAGE is 12 (on Fedora build machine 16; see pagesize above)
 * My test/unit/pack.c compiles warning free

So I guess the different pagesize is the problem.

Other architectures support different pagesizes as well ([1]),
but have a default of 4k in Fedora 25 (at least on amd64, x86).
This leads to the following question:
Why is jemalloc for aarch64 compiled on a system with a pagesize of 64k?

I can give it a try to change my kernel config to use a 64k pagesizes
as well (should not make a difference on my 16 GiB machine), but other
arm64 machines with less RAM (e.g. Pine64/Raspberry Pi3) might not want
to use such a big pagesize (and thus cannot use jemalloc if they run
Fedora 25).

Thanks,
Christoph

[1] https://wiki.debian.org/ArchitectureSpecificsMemo

Comment 6 Ingvar Hagelund 2017-01-30 10:23:57 UTC
Hello Christoph

The patch jemalloc-4.4.0.disable_thp.patch is only applied if the kernel on the system you build the package, do not support transparent hugepages.

I have no actual fix for this. The pagesize on all fedora kernels on the same platform should be the same, should it not? If it isn't, how can we configure memory allocators?

I don't think jemalloc can reconfigure the page size at run time, but I can check.

Peter, do you have any comments?

Ingvar

Comment 7 cmuellner 2017-01-30 12:47:48 UTC
Hi Ingvar,

over the weekend I've had a look at the default pagesize on several aarch64
Linux distributions. It seems like only Fedora has a pagesize which is not 4k.
Which is not a problem, if the Fedora kernel (and its config) is used.
As I cannot use the Fedora kernel (some drivers for my machine are missing),
it is my responsibility to provide one, which is compatible to the one from Fedora.
In short: after recompiling my kernel with a pagesize of 64k, jemalloc
works fine on my machine.

Dynamically detecting the pagesize would lead to slower code being generated by gcc
as the 4k/64k constant fits perfectly fine into a load-immediate instruction
whereas a dynamic lookup would have to load the value from memory (and it might
not be cached). This is something I would not like to see either... ;)

So feel free to close this ticket and thank's for all your help!

BR
Christoph


Note You need to log in before you can comment on or make changes to this bug.