Bug 1914283

Summary: python-mne fails to install, because it fails to build with Python 3.10 due to test failures
Product: [Fedora] Fedora Reporter: Tomáš Hrnčiar <thrnciar>
Component: python-mneAssignee: Aniket Pradhan <aniketpradhan1999>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 35CC: aniketpradhan1999, awilliam, igor.raits, mhroncok, neuro-sig, sanjay.ankur, thrnciar
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: AcceptedFreezeException
Fixed In Version: python-mne-0.23.3-1.fc35 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-09-07 19:06:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1917391, 1945960    
Bug Blocks: 1890881, 1891954, 1927309, 1927313, 1969032, 1987874, 1992484, 1992487    

Description Tomáš Hrnčiar 2021-01-08 14:20:50 UTC
python-mne fails to build with Python 3.10.0a4.

=================================== FAILURES ===================================
_______________________________ test_object_size _______________________________
mne/utils/tests/test_numerics.py:305: in test_object_size
    assert lower < size < upper, \
E   AssertionError: 100 < 64 < 400:
E     {}
E   assert 100 < 64

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01865172-python-mne/

For all our attempts to build python-mne with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/python-mne/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Ben Cotton 2021-02-09 15:38:25 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 2 Miro Hrončok 2021-06-04 20:14:14 UTC
This is a mass-posted update. Sorry if it is not 100% accurate to this bugzilla.


The Python 3.10 rebuild is in progress in a Koji side tag. If you manage to fix the problem, please commit the fix in the rawhide branch, but don't build the package in regular rawhide.

You can either build the package in the side tag, with:

    $ fedpkg build --target=f35-python

Or you can the build and we will eventually build it for you.

Note that the rebuild is still in progress, so not all (build) dependencies of this package might be available right away.

Thanks.

See also https://fedoraproject.org/wiki/Changes/Python3.10

If you have general questions about the rebuild, please use this mailing list thread: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/G47SGOYIQLRDTWGOSLSWERZSSHXDEDH5/

Comment 3 Miro Hrončok 2021-06-07 22:58:54 UTC
The f35-python side tag has been merged to Rawhide. From now on, build as you would normally build.

Comment 4 Miro Hrončok 2021-06-08 11:28:26 UTC
*** Bug 1969031 has been marked as a duplicate of this bug. ***

Comment 5 Ankur Sinha (FranciscoD) 2021-07-06 18:16:16 UTC
Some of this seems to be fixed in Git, for example: https://github.com/mne-tools/mne-python/commit/e8727cb016d034263a4f17e68e12b1ebf5dacc8d

but it'll need to be backported to this release (doesn't apply cleanly).

Aniket, do you think this is worth doing, or should we package the current snapshot for the time being maybe? Thoughts?

Comment 6 Ben Cotton 2021-08-10 12:50:12 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Comment 7 Aniket Pradhan 2021-08-17 15:56:23 UTC
Waiting for Scipy to create a release with Py3.10 support. As soon as it is released, the next version of python-mne with py3.10 support can be provided.

https://mne.discourse.group/t/mne-python-new-release/3558/4

Comment 8 Miro Hrončok 2021-08-18 07:38:29 UTC
Note that SciPy already works with Python 3.10, it is just they have not yet released wheels.

Comment 9 Miro Hrončok 2021-08-24 14:09:00 UTC
I've tried to build the tarball from the unreleased branch, but I've ended up with:

==================================== ERRORS ====================================
_________________ ERROR collecting tools/generate_codemeta.py __________________
tools/generate_codemeta.py:61: in <module>
    all_names = [parse_name(line) for line in lines]
tools/generate_codemeta.py:61: in <listcomp>
    all_names = [parse_name(line) for line in lines]
tools/generate_codemeta.py:25: in parse_name
    _, name_and_email = name.strip().split('\t')
E   ValueError: not enough values to unpack (expected 2, got 1)

Comment 10 Aniket Pradhan 2021-08-24 18:56:18 UTC
Hey Miro,

Thanks for trying it out! :D

From what I could gather, the files in `tools/` are the helper files for the GitHub workflows and are not required for building the package (outside the GitHub env). Pytest also collects these files while collecting tests and runs them (since they are scripts). If these scripts were run within the Git directory, there wouldn't be an issue since it runs `git shortlog -nse` to get a list of the authors.

One could simply remove the files in `tools/` and manually build the package without any issues.

Comment 11 Miro Hrončok 2021-08-24 19:39:40 UTC
That leaves:

=================================== FAILURES ===================================
_____________________________ test_module_nesting ______________________________
mne/tests/test_import_nesting.py:43: in test_module_nesting
    assert code == 0, stdout + stderr
E   AssertionError: 
E     Found un-nested import(s) for ["scipy submodules: ['misc', 'sparse']"]/builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
E     scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
E       icount = indentcount_lines(lines[1:])
E     /builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
E     scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
E       icount = indentcount_lines(lines[1:])
E     /builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
...snip...
E     scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
E       icount = indentcount_lines(lines[1:])
E     /builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
E     scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
E       icount = indentcount_lines(lines[1:])
E     /builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
E     scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
E       icount = indentcount_lines(lines[1:])
E     
E   assert 1 == 0
----------------------------- Captured stdout call -----------------------------
Running subprocess: /usr/bin/python3 -c 
import sys
import mne
out = set()
# check scipy (Numba imports it to check the version)
ok_scipy_submodules = set(['scipy', 'numpy',  # these appear in old scipy
                           'version'])
scipy_submodules = set(x.split('.')[1] for x in sys.modules.keys()
                       if x.startswith('scipy.') and '__' not in x and
                       not x.split('.')[1].startswith('_')
                       and sys.modules[x] is not None)
bad = scipy_submodules - ok_scipy_submodules
if len(bad) > 0:
    out |= {'scipy submodules: %s' % list(bad)}
# check sklearn and others
for x in sys.modules.keys():
    for key in ('sklearn', 'pandas', 'mayavi', 'pyvista', 'matplotlib',
                'dipy', 'nibabel', 'cupy', 'picard', 'pyvistaqt'):
        if x.startswith(key):
            x = '.'.join(x.split('.')[:2])
            out |= {x}
if len(out) > 0:
    print('\nFound un-nested import(s) for %s' % (sorted(out),), end='')
exit(len(out))
/builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
  icount = indentcount_lines(lines[1:])
/builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
...snip...
/builddir/build/BUILD/mne-python-maint-0.23/mne/utils/docs.py:2329: DeprecationWarning: `indentcount_lines` is deprecated!
scipy.misc.indentcount_lines is deprecated in SciPy 1.3.0
  icount = indentcount_lines(lines[1:])
Found un-nested import(s) for ["scipy submodules: ['misc', 'sparse']"]

Comment 12 Aniket Pradhan 2021-08-24 19:50:21 UTC
Already reported upstream. Thanks for helping us out Miro :')

While we're at it, I was also trying to find a way to run the graphical tests. Although I tried out running a xvfb buffer but it somehow doesn't work in mock. It usually results in a segfault in mock but works fine on my system. Is there any way to diagnose the issue and find a solution for the same?

I had a similar issue earlier with some other tool with some graphical tests but I couldn't find a solution then and had to leave the tool unpackaged.

Comment 13 Miro Hrončok 2021-08-25 23:46:58 UTC
> Is there any way to diagnose the issue and find a solution for the same?

No idea. I can certainly have a look if you give me a spec file to try, but no promises.

Comment 14 Miro Hrončok 2021-08-30 16:50:59 UTC
https://src.fedoraproject.org/rpms/python-mne/pull-request/3
https://src.fedoraproject.org/rpms/python-mne/pull-request/4

Nominating as beta freeze exception, as the package does not even install otherwise.

Comment 15 Fedora Update System 2021-08-31 12:43:02 UTC
FEDORA-2021-357e8396cc has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-357e8396cc

Comment 16 Fedora Update System 2021-08-31 12:44:36 UTC
FEDORA-2021-844d3220ea has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2021-08-31 17:57:41 UTC
FEDORA-2021-357e8396cc has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-357e8396cc`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-357e8396cc

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 18 Aniket Pradhan 2021-09-04 18:11:46 UTC
Thanks for the PR Miro!

I'll still have to talk to upstream about their compatibility with armv7 support, because my scratch builds were failing on that arch for some reason. This is great, thanks for your help! :D

Comment 19 Adam Williamson 2021-09-06 15:09:20 UTC
+3 in https://pagure.io/fedora-qa/blocker-review/issue/418 , marking accepted.

Comment 20 Fedora Update System 2021-09-07 19:06:20 UTC
FEDORA-2021-357e8396cc has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.