Bug 1558091

Summary:	Different processor affinity for Fedora and julialang.org builds
Product:	[Fedora] Fedora	Reporter:	Jeroen Wouters <woutersj>
Component:	openblas	Assignee:	Susi Lehtola <susi.lehtola>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	27	CC:	nalimilan, susi.lehtola
Target Milestone:	---	Keywords:	Reopened
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openblas-0.2.20-10.fc28 openblas-0.2.20-10.fc27 openblas-0.2.20-6.el6 openblas-0.2.20-6.el7	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-03-30 13:15:04 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jeroen Wouters 2018-03-19 15:41:01 UTC

Description of problem:
Calling an external mpi executable from julia behaves differently in the julia builds from the Fedora repository and those downloaded from julialang.org. The official builds have the NO_AFFINITY option set for BLAS and calling an external mpi executable by
> cmd = Cmd(`mpiexec -np 4 name_of_executable`)
> run(cmd)
runs the executable on 4 cores.

Doing the same in the Fedora build, creates 4 processes running on one core.


Version-Release number of selected component (if applicable):

versioninfo() for Fedora build:
Julia Version 0.6.0
Commit 903644385b* (2017-06-19 13:05 UTC)
Platform Info:
  OS: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH Haswell)
  LAPACK: libopenblasp.so.0
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

versioninfo() for official build:
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Prescott)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)


How reproducible:
Always

Steps to Reproduce:
1. Run julia
2. execute
> cmd = Cmd(`mpiexec -np 4 name_of_executable`)
> run(cmd)

Actual results:
4 processes are created, all running on the same core

Expected results:
4 processes are created, all running on their own core

Additional info:

Comment 1 Milan Bouchet-Valat 2018-03-19 16:24:08 UTC

Thanks for the report. Shouldn't this be fixed in OpenBLAS rather than in Julia? What's the role of Julia in this?

Also, AFAICT Julia disables threading affinity on startup via OPENBLAS_MAIN_FREE https://github.com/JuliaLang/julia/pull/9639.

Comment 2 Milan Bouchet-Valat 2018-03-19 16:34:10 UTC

FWIW, NO_AFFINITY=1 has been the default in a few OpenBLAS releases, but has been reverted without a clear reason. Julia developers argue that distribution packages for OpenBLAS should set NO_AFFINITY=1. That would protect us from future upstream changes. See https://github.com/JuliaLang/julia/pull/9639

Comment 3 Milan Bouchet-Valat 2018-03-19 17:10:21 UTC

Fixed upstream in OpenBLAS: https://github.com/xianyi/OpenBLAS/pull/1495. May still be worth passing the flag explicitly to fix the problem immediately instead of waiting for the next release.

Comment 4 Fedora Update System 2018-03-22 19:17:42 UTC

openblas-0.2.20-6.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-c53bb31492

Comment 5 Fedora Update System 2018-03-22 19:17:48 UTC

openblas-0.2.20-6.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-a981a0cbd7

Comment 6 Fedora Update System 2018-03-22 19:17:53 UTC

openblas-0.2.20-10.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-56401d52db

Comment 7 Fedora Update System 2018-03-22 19:17:59 UTC

openblas-0.2.20-10.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-493ca8ae30

Comment 8 Fedora Update System 2018-03-23 14:44:19 UTC

openblas-0.2.20-10.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-56401d52db

Comment 9 Milan Bouchet-Valat 2018-03-23 16:10:59 UTC

Thanks!

Comment 10 Fedora Update System 2018-03-23 17:24:42 UTC

openblas-0.2.20-6.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-a981a0cbd7

Comment 11 Fedora Update System 2018-03-23 17:46:44 UTC

openblas-0.2.20-10.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-493ca8ae30

Comment 12 Fedora Update System 2018-03-23 18:30:24 UTC

openblas-0.2.20-6.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-c53bb31492

Comment 13 Fedora Update System 2018-03-30 13:15:04 UTC

openblas-0.2.20-10.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 14 Fedora Update System 2018-04-01 03:28:43 UTC

openblas-0.2.20-10.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 15 Fedora Update System 2018-04-10 17:49:45 UTC

openblas-0.2.20-6.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2018-04-10 18:07:04 UTC

openblas-0.2.20-6.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.