Bug 1615803

Summary:	After update to openblas.x86_64 0.3.1-2.fc28 previously working code crashes
Product:	[Fedora] Fedora	Reporter:	Paul DeStefano <prd-fedora>
Component:	openblas	Assignee:	Susi Lehtola <susi.lehtola>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	28	CC:	susi.lehtola
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-01-08 08:09:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Paul DeStefano 2018-08-14 09:42:39 UTC

Description of problem:
I had working python script a couple months ago.  I upgraded openblas RPM a week ago and now the script crashes with multiple warnings:

OpenBLAS : Program will terminate because you tried to start too many threads.
OpenBLAS : Program will terminate because you tried to start too many threads.
OpenBLAS : Program will terminate because you tried to start too many threads.
OpenBLAS : Program will terminate because you tried to start too many threads.
OpenBLAS : Program will terminate because you tried to start too many threads.
Segmentation fault (core dumped)


Version-Release number of selected component (if applicable):
openblas-0.3.1-2.fc28.x86_64

How reproducible:
Well, it doesn't happen immediately, but, as the script runs, it inevitably crashes. The number of termination notices seems to be growing.

I don't know how to troubleshoot this as I am not intentionally using OpenBLAS and the crash doesn't produce a traceback.

Additional info:
None of my Python code has anything to do with threads, but I do use numpy, scipy, pandas, uncertainties, matplotlib, and multiprocessing.

Comment 1 Susi Lehtola 2018-08-14 09:55:00 UTC

"I upgraded openblas RPM a week ago"

Please run dnf update; you're filing a bug against an old version of the package.

Comment 2 Paul DeStefano 2018-08-14 15:50:08 UTC

Argh!  Same results with openblas-0.3.2-1.fc28.x86_64.

This is serious.  I don't know what to do and I need this code to work very very soon.

Comment 3 Susi Lehtola 2018-08-14 21:20:00 UTC

Looks like this is an active ticket upstream
https://github.com/xianyi/OpenBLAS/issues/1735

Unfortunately it appears that the testing that OpenBLAS upstream performs is not as extensive as one would like...

Comment 4 Paul DeStefano 2018-08-15 00:36:15 UTC

Ah, okay.  Thanks for your help!

Always willing to troubleshoot, if you have any ideas.

Using env var OPENBLAS_NUM_THREADS=1, the code doesn't crash.  Does that help?  Probably not.

I don't know why python would use OpenBLAS since I'm not doing any lin algebra, directly.  The code seems to crash when I write a MPL plot to a file, but I don't know if that is coincident with the error or if the problem occurs before or after that point.  The error message is useless to me.

Comment 5 Susi Lehtola 2018-08-15 10:18:18 UTC

OPENBLAS_NUM_THREADS=1 is tantamount to disabling parallellization, so it's not surprising you're experiencing fewer issues.

Something in your code is doing linear algebra. Numpy probably links to OpenBLAS.

I'll wait for upstream to resolve this.

Comment 6 Susi Lehtola 2019-01-08 08:09:40 UTC

I think this was solved in 0.3.3 several months ago.