Bug 1118734 - RFE: compute the value for smp_mflags from number of cpus and available memory
Summary: RFE: compute the value for smp_mflags from number of cpus and available memory
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-07-11 12:24 UTC by Dan Horák
Modified: 2019-08-05 07:50 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Dan Horák 2014-07-11 12:24:55 UTC
I would like to suggest to change how the value for smp_mflags is computed. Currently it is the number of cpus online and 16 is the max. I suggest to include the memory size into the calculation, because with for example C++ code the compiler can easily require 2GB per process. So if the builder machine doesn't have adequate memory it starts swapping and later the compiler (or linker) gets killed by the kernel OOM killer.

my proposal
smp_mflags = MIN( getconf _NPROCESSORS_ONLN , RAMSIZE / TASKSIZE)
and use eg. TASKSIZE=2GB

The real case where OOM is in action during builds is on ppc64 builders with 48 (shared virtual) CPUs and 10GB of memory (+ some swap) and building ceph. I think it can also help on ARM with 4 CPUs and only 4GB of memory when they start swapping easily.

Comment 1 Fedora Admin XMLRPC Client 2014-11-14 07:24:40 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 2 Panu Matilainen 2019-03-07 10:46:50 UTC
%_smp_mflags lives in rpm now, switching component (wow this is an old one...)

It'd need some tunables (tasksize at least) and maybe 32/64bit-ness should be taken into account for the heuristics too.

Comment 3 Dan Horák 2019-08-05 07:50:42 UTC
Ack, tunable "tasksize" makes sense.


Note You need to log in before you can comment on or make changes to this bug.