Bug 1871994

Summary: Parallel loops with joblib-0.16.0 doesn't work with Python3.9 on F33
Product: [Fedora] Fedora Reporter: Aniket Pradhan <aniketpradhan1999>
Component: python-joblibAssignee: Sergio Pascual <sergio.pasra>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 33CC: besser82, cstratak, igor.raits, sergio.pasra
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-joblib-0.16.0-3.fc34 python-joblib-0.16.0-3.fc33 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-09 09:20:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1837012, 1871604    
Attachments:
Description Flags
error output none

Description Aniket Pradhan 2020-08-24 19:52:34 UTC 
Created attachment 1712432 [details]
error output

Created attachment 1712432 [details]
error output

Created attachment 1712432 [details]
error output

Description of problem: I tried a simple example from joblib docs about parallel loops using multiprocessing, and I got a whole bunch of errors instead of the expected output.

Since, I tried an example from the documentation, it should have provided me with a correct output.


Version-Release number of selected component (if applicable): 0.16.0-1


How reproducible: Just tried this after updating my system to F33 on Python3.9.0rc1


Steps to Reproduce:
1. Copy and paste this simple example from the documentation (https://joblib.readthedocs.io/en/latest/parallel.html)

python -c "from math import sqrt; from joblib import Parallel, delayed; Parallel(n_jobs=2)(delayed(sqrt)(i ** 2) for i in range(10))"


Actual results: Refer to the attached file.


Expected results: [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]
Comment 1 Aniket Pradhan 2020-08-25 03:57:59 UTC 
The issue is already accounted for in the joblib spec.

> # Loky's _SafeQueue is not compatible with multiprocessing
> # of Python 3.9 nightly
> # https://github.com/joblib/loky/issues/251
https://src.fedoraproject.org/rpms/python-joblib/blob/master/f/python-joblib.spec#_55
Comment 2 Sergio Pascual 2020-08-25 21:30:42 UTC 
(In reply to Aniket Pradhan from comment #1)
> The issue is already accounted for in the joblib spec.
> 
> > # Loky's _SafeQueue is not compatible with multiprocessing
> > # of Python 3.9 nightly
> > # https://github.com/joblib/loky/issues/251
> https://src.fedoraproject.org/rpms/python-joblib/blob/master/f/python-joblib.
> spec#_55

Is the same problem? _SafeQueue problem is still not addressed by upstream
Comment 3 Aniket Pradhan 2020-08-26 11:50:42 UTC 
> Is the same problem? _SafeQueue problem is still not addressed by upstream

It could be related to the same problem, I am not sure. The issue seems to be with the the loky backend undergoing a segmentation fault. I'll report this to upstream joblib.
Comment 4 Charalampos Stratakis 2020-09-07 17:36:26 UTC 
PR: https://src.fedoraproject.org/rpms/python-joblib/pull-request/2

This also fixes rhbz#1837012
Comment 5 Fedora Update System 2020-09-09 09:20:03 UTC 
FEDORA-2020-69bd4b5bf0 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 6 Fedora Update System 2020-09-09 15:53:17 UTC 
FEDORA-2020-4f07527b3e has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-4f07527b3e
Comment 7 Fedora Update System 2020-09-10 15:48:36 UTC 
FEDORA-2020-4f07527b3e has been pushed to the Fedora 33 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-4f07527b3e`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-4f07527b3e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 8 Fedora Update System 2020-09-10 17:43:59 UTC 
FEDORA-2020-4f07527b3e has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-4f07527b3e
Comment 9 Fedora Update System 2020-09-14 23:44:06 UTC 
FEDORA-2020-4f07527b3e has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.