Bug 1705442

Summary: rmol fails to rebuild with Python 3.8
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: rmolAssignee: Denis Arnaud <denis.arnaud_fedora>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: rawhideCC: denis.arnaud_fedora
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-11 22:23:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1686977    
Attachments:
Description Flags
Full log from Copr none

Description Miro Hrončok 2019-05-02 10:27:02 UTC
Created attachment 1561492 [details]
Full log from Copr

When I try to rebuild rmol 1.00.2-5.fc31 for Python 3.8, I get:

!  ==> Fatal error occurred, no output PDF file produced!
...
make[2]: Leaving directory '/builddir/build/BUILD/rmol-rmol-1.00.2'
make[1]: *** [CMakeFiles/Makefile2:453: rmol/CMakeFiles/rmollib.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.5nLwdw (%build)

Full log attached.

This might not be Python 3.8 related, but if the package does not get a rebuild, it won't install in Fedora 31 once we do the Python update.

Comment 1 Miro Hrončok 2019-07-10 10:09:54 UTC
Please respond.

Comment 2 Denis Arnaud 2019-07-10 14:56:01 UTC
Thanks for the follow up (and the initial bug report)!

Similarly to BZ#1705441, I'll have a look as soon as possible.

Comment 3 Denis Arnaud 2019-07-10 18:08:18 UTC
(As expected,) both rmol and trademgen build on Rawhide. Would you be so kind as to tell me how I can test the build with Python 3.8?
You can just point to Docker/LXC image, or how to activate Python 3.8 on Rawhide.

Comment 4 Miro Hrončok 2019-07-10 22:15:12 UTC
Sure:

For all our attempts to build rmol with Python 3.8, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.8/package/rmol/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.8:
https://copr.fedorainfracloud.org/coprs/g/python/python3.8/

Let us know here if you have any questions.

Comment 5 Denis Arnaud 2019-07-11 17:13:26 UTC
Trying to reproduce the issue in a way I can interact with the source code and the build environment, I came up with the following (for the record):
# On a Rawhide system
$ dnf copr enable @python/python3.8
$ dnf -y update && dnf -y install python38-devel && dnf reinstall -y python3
$ python --version
3.8.0
$ mkdir ~/tmp && cd ~/tmp && fedpkg clone rmol && cd rmol && mkdir build && cd build
$ cmake -DCMAKE_C_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_Fortran_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_INSTALL_PREFIX:PATH=/usr -DINCLUDE_INSTALL_DIR:PATH=/usr/include -DLIB_INSTALL_DIR:PATH=/usr/lib64 -DSYSCONF_INSTALL_DIR:PATH=/etc -DSHARE_INSTALL_PREFIX:PATH=/usr/share -DLIB_SUFFIX=64 -DBUILD_SHARED_LIBS:BOOL=ON ..

That build fails on the fact that a dependency (namely stdair, here) itself exports libpython3.7m.so as a dependency, which cannot be found on a Python 3.8 system. So, the first step is to rebuild stdair. Which I just did (https://koji.fedoraproject.org/koji/taskinfo?taskID=36189978), updating along the way the (upstream) source code with new CMake support files, improving the overall compatibility with Python 3.

So, the problem (for rmol and trademgen) may come from the fact that their common dependency, namely stdair, has not been rebuilt for Python 3.8: https://copr.fedorainfracloud.org/coprs/g/python/python3.8/package/stdair/ does not show any rebuild for stdair.
$ dnf -y install stdair-devel
$ grep -Hin "python" /usr/share/stdair/CMake/*.cmake
/usr/share/stdair/CMake/stdair-library-depends.cmake:51:  INTERFACE_LINK_LIBRARIES "/usr/lib64/libpython3.7m.so;/usr/lib64/libboost_date_time.so;/usr/lib64/libboost_random.so;/usr/lib64/libboost_iostreams.so;/usr/lib64/libboost_serialization.so;/usr/lib64/libboost_filesystem.so;/usr/lib64/libboost_system.so;/usr/lib64/libboost_locale.so;/usr/lib64/libboost_python37.so;/usr/lib64/libboost_regex.so;/usr/lib64/libmariadb.so;/usr/lib64/libsoci_core.so;/usr/lib64/libsoci_mysql.so;/usr/lib64/libsoci_core.so;/usr/lib64/libsoci_sqlite3.so;/usr/lib64/libreadline.so"
/usr/share/stdair/CMake/stdair-library-depends.cmake:58:  INTERFACE_LINK_LIBRARIES "/usr/lib64/libpython3.7m.so;/usr/lib64/libboost_date_time.so;/usr/lib64/libboost_random.so;/usr/lib64/libboost_iostreams.so;/usr/lib64/libboost_serialization.so;/usr/lib64/libboost_filesystem.so;/usr/lib64/libboost_system.so;/usr/lib64/libboost_locale.so;/usr/lib64/libboost_python37.so;/usr/lib64/libboost_regex.so;/usr/lib64/libmariadb.so;/usr/lib64/libsoci_core.so;/usr/lib64/libsoci_mysql.so;/usr/lib64/libsoci_core.so;/usr/lib64/libsoci_sqlite3.so;/usr/lib64/libreadline.so;stdairlib"

And we have the same issue with airrac.

In short, I would recommend to rebuild all the dependencies of rmol and trademgen, in the right order:
1. stdair
2. airrac
3. rmol
4. sevmgr
5. trademgen

Then, the other packages depending on those should also be rebuilt:
6. airtsp
7. simfqt
8. travelccm
9. airinv
10. simcrs

How can we add the missing packages to the COPR Python3.8 repository/builds (https://copr.fedorainfracloud.org/coprs/g/python/python3.8/packages/)?

Comment 6 Miro Hrončok 2019-07-11 17:44:31 UTC
What is python38-devel ?

Anyway, let me check the dependencies of stdair:

$ repoquery --repo=rawhide --requires stdair | grep python
...nothing...

Or airrac:

$ repoquery --repo=rawhide --requires airrac | grep python
...nothing...

I don't understand the problem, sorry.

Comment 7 Miro Hrončok 2019-07-11 17:46:44 UTC
The copr has all source packages that produce packages that depend on either of the following:

libpython3.7m.so.1.0()(64bit)
python(abi) = 3.7

Comment 8 Denis Arnaud 2019-07-11 20:44:12 UTC
(In reply to Miro Hrončok from comment #6)
> What is python38-devel ?
> 

Well, python3-devel with Python 3.8


> Anyway, let me check the dependencies of stdair:
> 
> $ repoquery --repo=rawhide --requires stdair | grep python
> ...nothing...
> 
> Or airrac:
> 
> $ repoquery --repo=rawhide --requires airrac | grep python
> ...nothing...
> 
> I don't understand the problem, sorry.
>

Both stdair and airrac, which are required by rmol (and stdair being needed by trademgen), are providing a CMake support file (respectively /usr/share/stdair/CMake/stdair-library-depends.cmake and /usr/share/airrac/CMake/airrac-library-depends.cmake) where libpython3.7m.so is explicitly referenced, as mentioned above. That is certainly a bug of stdair and airrac (to state that they require libpython3.7m.so, whereas they do not need Python), but that CMake support file nevertheless induces rmol and trademgen to require libpython3.7m.so.

So, the solution is easy: stdair, airrac and sevmgr need to be rebuilt in the Python3.8 COPR environment.

If you prefer, you can just remove those packages (trademgen and rmol) from COPR Python3.8, and when Fedora 31 will switch to Python 3.8, FTBS bugs will be created for those two packages. I will then fix those, no worry.

Comment 9 Miro Hrončok 2019-07-11 21:03:23 UTC
oh. so the libpython3.7m string is compiled in on build time?

I suggest to add:

    Requires: (python(abi) = %{python3_version} if python3-devel)

To stdair-devel and airrac-devel.


Running the stdair build.

Comment 10 Miro Hrončok 2019-07-11 22:08:49 UTC
The stdair and airrac builds succeeded, running the rmol build.

Comment 11 Miro Hrončok 2019-07-11 22:23:01 UTC
Thanks, rmol built.

Opened https://src.fedoraproject.org/rpms/stdair/pull-request/1 as a followup.

Comment 12 Denis Arnaud 2019-07-11 23:17:26 UTC
Many thanks, both for your quick actions/follow up and for the merge request/suggestion!