Bug 1903952

Summary: espresso fails to build with Python 3.10: some tests timeouted
Product: [Fedora] Fedora Reporter: Tomáš Hrnčiar <thrnciar>
Component: espressoAssignee: Christoph Junghans <junghans>
Status: CLOSED WORKSFORME QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: dakingun, jgrad, junghans, mhroncok, thrnciar
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-06-09 16:55:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1890881, 1927309, 1927313    

Description Tomáš Hrnčiar 2020-12-03 08:40:28 UTC
espresso fails to build with Python 3.10.0a2.

The following tests FAILED:
	 25 - correlation (Timeout)
	 50 - rigid_bond (Timeout)
	 52 - rotational_inertia (Timeout)
	 55 - reaction_ensemble (Timeout)
	 57 - constant_pH (Timeout)
	 78 - dpd (Timeout)
	 85 - collision_detection (Timeout)
	 98 - npt (Timeout)
Errors while running CTest
gmake[3]: *** [testsuite/python/CMakeFiles/check_python.dir/build.make:79: testsuite/python/CMakeFiles/check_python] Error 8
gmake[3]: Leaving directory '/builddir/build/BUILD/espresso/mpich'
gmake[2]: Leaving directory '/builddir/build/BUILD/espresso/mpich'
gmake[2]: *** [CMakeFiles/Makefile2:1225: testsuite/python/CMakeFiles/check_python.dir/all] Error 2
gmake[1]: Leaving directory '/builddir/build/BUILD/espresso/mpich'
gmake[1]: *** [CMakeFiles/Makefile2:1177: CMakeFiles/check.dir/rule] Error 2
gmake: *** [Makefile:197: check] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.JalCmv (%check)
    Bad exit status from /var/tmp/rpm-tmp.JalCmv (%check)

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01806934-espresso/

For all our attempts to build espresso with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/espresso/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Ben Cotton 2021-02-09 15:29:39 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 2 Jean-Noël Grad 2021-02-10 21:13:58 UTC
I can confirm that ESPResSo 4.1 had extremely slow tests, which caused frequent time outs in our own CI infrastructure, in particular:
- wang_landau_reaction_ensemble
- reaction_ensemble
- constant_pH
- widom_insertion
- elc_vs_analytic
- dpd
- lb
- npt
- langevin
- coulomb_tuning
- mass-and-rinertia_per_particle
- rotational-diffusion-aniso
- lb_pressure_tensor

The CTest log excerpt in the first post shows extra tests, but this might just be a cascade effect, if the testsuite is running in parallel. A test that is timing out will lock CPU resources for the duration of the timeout, causing other tests running at the same time to slow down considerably, sometimes to the point of timing out too. This happened to us in the past, when computationally demanding tests like constant_pH, reaction_ensemble or wang_landau_reaction_ensemble timed out (we allocate only 2 CPU cores to run the testsuite in CI).

We have addressed this issue in the development branch of ESPResSo by simplifying tests, running them with fewer integration steps, or splitting them into smaller tests. This effort has been carried out over a period of several months for the upcoming ESPResSo 4.2 release. The last change in that project was https://github.com/espressomd/espresso/pull/3999. A couple of tests are still taking a minute to run but can be disabled thanks to the introduction of a CTest label that flags slow test.

The changes made in the testsuite are significant and cannot be backported to the 4.1.4 release. I would suggest disabling the slowest tests when building the 4.1.4 release, for example by commenting out the wang_landau_reaction_ensemble.py, reaction_ensemble.py, constant_pH.py and npt.py tests. Runtimes of the slowest tests can be found in https://github.com/espressomd/espresso/issues/3883, although these values have been measured near the end of the testsuite refactoring project and as such, do not fully reflect the state of 4.1.4.

Comment 3 Miro Hrončok 2021-06-04 20:14:57 UTC
This is a mass-posted update. Sorry if it is not 100% accurate to this bugzilla.


The Python 3.10 rebuild is in progress in a Koji side tag. If you manage to fix the problem, please commit the fix in the rawhide branch, but don't build the package in regular rawhide.

You can either build the package in the side tag, with:

    $ fedpkg build --target=f35-python

Or you can the build and we will eventually build it for you.

Note that the rebuild is still in progress, so not all (build) dependencies of this package might be available right away.

Thanks.

See also https://fedoraproject.org/wiki/Changes/Python3.10

If you have general questions about the rebuild, please use this mailing list thread: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/G47SGOYIQLRDTWGOSLSWERZSSHXDEDH5/

Comment 4 Miro Hrončok 2021-06-07 22:59:28 UTC
The f35-python side tag has been merged to Rawhide. From now on, build as you would normally build.

Comment 5 Miro Hrončok 2021-06-08 11:14:42 UTC
*** Bug 1968800 has been marked as a duplicate of this bug. ***

Comment 6 Miro Hrončok 2021-06-09 16:55:09 UTC
Hello,

Please note that this comment was generated automatically. If you feel that this output has mistakes, please contact me via email (mhroncok).

All subpackages of a package against which this bug was filled are now installable or removed from Fedora 35.

Thanks for taking care of it!