Bug 1118734

Summary: RFE: compute the value for smp_mflags from number of cpus and available memory
Product: [Fedora] Fedora Reporter: Dan Horák <dan>
Component: rpmAssignee: Panu Matilainen <pmatilai>
Status: ASSIGNED --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: debarshir, i.gnatenko.brain, jonathan, ktdreyer, mjw, packaging-team-maint, pmatilai, pmoravco, vmukhame
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Dan Horák 2014-07-11 12:24:55 UTC
I would like to suggest to change how the value for smp_mflags is computed. Currently it is the number of cpus online and 16 is the max. I suggest to include the memory size into the calculation, because with for example C++ code the compiler can easily require 2GB per process. So if the builder machine doesn't have adequate memory it starts swapping and later the compiler (or linker) gets killed by the kernel OOM killer.

my proposal
smp_mflags = MIN( getconf _NPROCESSORS_ONLN , RAMSIZE / TASKSIZE)
and use eg. TASKSIZE=2GB

The real case where OOM is in action during builds is on ppc64 builders with 48 (shared virtual) CPUs and 10GB of memory (+ some swap) and building ceph. I think it can also help on ARM with 4 CPUs and only 4GB of memory when they start swapping easily.

Comment 1 Fedora Admin XMLRPC Client 2014-11-14 07:24:40 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 2 Panu Matilainen 2019-03-07 10:46:50 UTC
%_smp_mflags lives in rpm now, switching component (wow this is an old one...)

It'd need some tunables (tasksize at least) and maybe 32/64bit-ness should be taken into account for the heuristics too.

Comment 3 Dan Horák 2019-08-05 07:50:42 UTC
Ack, tunable "tasksize" makes sense.

Comment 4 Panu Matilainen 2019-08-28 11:32:38 UTC
FWIW, there's a work-in-progress PR at upstream where further input (+ testing) is welcome:
https://github.com/rpm-software-management/rpm/pull/821