Bug 1529001

Summary: RFE: Reorganize the pypy3 package to be more sane + cleanup the spec file
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: pypy3.9Assignee: Python Maintainers <python-maint>
Status: CLOSED RAWHIDE QA Contact:
Severity: low Docs Contact:
Priority: low    
Version: rawhideCC: lbalhar, liling, mhroncok, pviktori, python-maint, python-sig, thrnciar, torsava
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-01 16:25:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
arrange lib64 and include files closer to those of python3.9 none

Description Miro Hrončok 2017-12-25 19:59:42 UTC
The specfile has plenty of archaic bloat. It has ~800 lines without the %changelog. I believe it can be simplified. (Maybe together with pypy2.)

Comment 1 Lumír Balhar 2019-05-06 11:14:36 UTC
I took a look and I don't think that it's worth to invest time to that. Moreover, in mentioned more than 800 lines there are more than 220 lines of comments and 40 lines of echos. I don't think it's necessary but the very first step of specfile simplification might be to move BuildPyPy() and CheckPyPy() functions to separated files.

Comment 2 Petr Viktorin 2020-06-24 12:32:07 UTC
This is not just about how the specfile is written, but also the semantics -- things like where files get installed, etc.

Comment 3 Fedora Admin user for bugzilla script actions 2021-05-14 00:03:29 UTC
This package has changed maintainer in Fedora. Reassigning to the new maintainer of this component.

Comment 4 Miro Hrončok 2021-05-26 12:00:08 UTC
Things that bother me:

- The package split between pypy3, pypy3-libs and pypy3-devel is inconsistent with the CPython packages, e.g. the test are not split at all or the libpypy3-c.so file is in the main package.
- Many weird files are installed, such as .c sources etc.
- READMEs and LICENSEs are not marked as such.
- The filesystem namespace is /usr/lib64/pypy3-7.3/ hence we cannot have PyPy 3.6 and PyPy 3.7 as co-installable packages.

I think the latest point could be solved by simply renaming the package to pypy3.7, which is good. That's what I'll explore first.

Comment 5 Tomáš Hrnčiar 2021-08-11 11:57:33 UTC
(In reply to Miro Hrončok from comment #4)
> - The filesystem namespace is /usr/lib64/pypy3-7.3/ hence we cannot have
> PyPy 3.6 and PyPy 3.7 as co-installable packages.
> 
> I think the latest point could be solved by simply renaming the package to
> pypy3.7, which is good. That's what I'll explore first.

This part is finished in F35+, we created a new component pypy3.7 that installs to /usr/lib64/pypy3.7 .

Comment 6 Tomas Orsava 2021-12-07 10:14:39 UTC
We're waiting for pypy3.9 to come out with changes of the file layout to resemble CPython more to finish this change.

Comment 7 Miro Hrončok 2022-02-01 10:06:12 UTC
https://src.fedoraproject.org/rpms/pypy3.8/pull-request/2 is a pypy3.9 WIP PR.

I'll use it to reorganize subpackges to be more or less like python3.9, remove years-unused %if branches, and outdated comments.

Comment 8 Ling Li 2022-02-07 22:11:27 UTC
Created attachment 1859666 [details]
arrange lib64 and include files closer to those of python3.9

May I also suggest changing the file layout to be closer to those of python39?  I.e., 

   /usr/lib64/pypy3.9/lib64/pypy3.9 -> /usr/lib64/pypy3.9
   /usr/lib64/pypy3.9/include/pypy3.9 -> /usr/include/pypy3.9

Based on Miro's pypy3.9 branch, I made some extra changes (see the attached patch) so that I can build pypy3.9 for EPEL 8.  The new layout seems working fine for me with a locally built RPM (I wish I could provide a copr build for you to verify, but always got "out of memory" when building the RPM on copr).

Comment 9 Miro Hrončok 2022-02-07 22:54:19 UTC
Thanks. I had that planned, yes. Including the -fno-semantic-interposition thing.

But there will be more stuff, see https://mail.python.org/pipermail/pypy-dev/2022-January/016310.html and the replies there. My plan was to finish this once the final 7.3.8 is out. 

This part is likely not Fedora compatible: 

 %if %{with rpmwheels}
-BuildRequires: python-setuptools-wheel
-BuildRequires: python-pip-wheel
+BuildRequires: python%{python3_pkgversion}-setuptools-wheel
+BuildRequires: python%{python3_pkgversion}-pip-wheel
 %endif

Have a look at https://src.fedoraproject.org/rpms/python3.10/pull-request/75

Comment 10 Ling Li 2022-02-07 23:20:04 UTC
That's very nice.  Really look forward to it when the final 7.3.8 is out.

(and please ignore any Fedora-incompatible bits; that was meant for my own hacky test on EPEL 8, and I wasn't sure whether it would work on Fedora)

Comment 11 Miro Hrončok 2022-02-10 14:51:59 UTC
https://src.fedoraproject.org/rpms/pypy3.8/pull-request/2 is ready for review

I wonder if I should add -fno-semantic-interposition.

Ling, have you tested if -fno-semantic-interposition makes PyPy faster?

Comment 12 Ling Li 2022-02-10 17:29:20 UTC
Sorry I haven't tested.  I am actually new to pypy, just started the exploration.

Can I ask you for a favor --- can you submit a build on COPR?  For the past few days I didn't have any luck in building pypy3.8 or pypy3.9 on COPR.  My local build always succeeds, but the copr builds failed on "cannot allocate memory".  I have seen your (temporary now deleted) builds were successful, and wonder if my failure is due to OS/compiler (GCC 8.5 vs GCC 11 on f35) or due to some changes in copr build vm (https://pagure.io/copr/copr/issue/2077).

Comment 13 Ling Li 2022-02-10 17:31:19 UTC
Btw, the changes in https://src.fedoraproject.org/rpms/pypy3.8/pull-request/2 look nice.

Comment 14 Miro Hrončok 2022-02-10 18:59:25 UTC
https://copr.fedorainfracloud.org/coprs/churchyard/pypy3.9/builds/ but it might fail as well

Comment 15 Ling Li 2022-02-11 00:40:52 UTC
Thank you.  Your builds all succeeded after 4 hours.  Initially I thought they would fail, because I noticed that they ran as slow as mine.  Usually it took 1-1.5 hours to finish, but mine (and the builds you just ran) would still be in the "translate" stage after two hours.  That's why I thought they would fail as well.  But they succeeded.  Thanks for helping with a good data point!

Comment 16 Miro Hrončok 2022-03-01 16:25:11 UTC
pypy3.9 is in rawhide with updates in testing. I consider the package quite sane.