Bug 2209055

Summary: RFE: namespace pyproject-rpm-macros generated text files with %{python3_pkgversion}
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: pyproject-rpm-macrosAssignee: Miro Hrončok <mhroncok>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: maxwell, mhroncok, pviktori, python-packagers-sig, torsava
Target Milestone: ---Keywords: RFE
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: pyproject-rpm-macros-1.12.0-1.fc40 pyproject-rpm-macros-1.12.0-1.fc39 pyproject-rpm-macros-1.12.0-1.fc38 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-01-26 10:12:13 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
A reproducer spec (for c9s) with an extension module none

Description Miro Hrončok 2023-05-22 12:41:44 UTC
Consider a RHEL 9 spec file with does things like:

    %generate_buildrequires
    %global python3_pkgversion 3
    %pyproject_buildrequires
    %global python3_pkgversion 3.11
    %pyproject_buildrequires

    %build
    %global python3_pkgversion 3
    %pyproject_wheel
    %global python3_pkgversion 3.11
    %pyproject_wheel

    %install
    %global python3_pkgversion 3
    %pyproject_install
    %pyproject_save_files module
    %global python3_pkgversion 3.11
    %pyproject_install
    %pyproject_save_files module

This should work, but it will blow up at the install stage:

Traceback (most recent call last):
  File "/usr/lib/rpm/redhat/pyproject_save_files.py", line 775, in <module>
    main(cli_args)
  File "/usr/lib/rpm/redhat/pyproject_save_files.py", line 730, in main
    file_section, module_names = pyproject_save_files_and_modules(
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/rpm/redhat/pyproject_save_files.py", line 708, in pyproject_save_files_and_modules
    parsed_records = load_parsed_record(pyproject_record)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/rpm/redhat/pyproject_save_files.py", line 676, in load_parsed_record
    raise FileExistsError("%pyproject install has found more than one *.dist-info/RECORD file. "
FileExistsError: %pyproject install has found more than one *.dist-info/RECORD file. Currently, %pyproject_save_files supports only one wheel → one file list mapping. Feel free to open a bugzilla for pyproject-rpm-macros and describe your usecase.


Even if we manage to fix this somehow, there are 2 wheels built in pyproject-wheeldir when the Python project contains an extension module:

    $ ls -1 /var/lib/mock/centos-stream+epel-next-9-x86_64/root/builddir/build/BUILD/pyproject-wheeldir/
    reproducer_double-1-cp39-cp39-linux_x86_64.whl
    reproducer_double-1-cp311-cp311-linux_x86_64.whl

And the %pyproject_insatll macro will likely try to install them both or blow up differently.

--------------

I propose we include %{python3_pkgversion} in:

 - %{_pyproject_wheeldir}
 - %{_pyproject_builddir}
 - %{_pyproject_files_prefix}

That should isolate the entire state between different %{python3_pkgversion} values.

If we want to make this 100% backwards compatible, we could only include %{python3_pkgversion} when it is not 3.

Comment 1 Miro Hrončok 2023-05-22 12:42:31 UTC
Created attachment 1966164 [details]
A reproducer spec (for c9s) with an extension module

Comment 2 Miro Hrončok 2023-05-22 12:51:07 UTC
Something like this should do:


%_pyproject_files_pkgversion %{expr:"%{python3_pkgversion}" != "3" ? "-%{python3_pkgversion}" : ""}

%_pyproject_wheeldir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/pyproject-wheeldir%{_pyproject_files_pkgversion}
%_pyproject_builddir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/.pyproject-builddir%{_pyproject_files_pkgversion}
%_pyproject_files_prefix %{name}-%{version}-%{release}.%{_arch}%{_pyproject_files_pkgversion}

Comment 3 Maxwell G 2023-05-31 16:42:05 UTC
I don't think this will work properly now that we've memoized a bunch of the `%python3_...` macros.

Comment 4 Miro Hrončok 2023-05-31 16:52:11 UTC
Yeah, the memoization will make this tricky. I was thinking a global way of disabling it and/or using %{python3_pkgversion} as a key in some  Lua table.

Comment 5 Tomas Orsava 2023-06-07 12:28:56 UTC
Related PR: https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/161

Comment 7 Fedora Update System 2024-01-26 10:11:46 UTC
FEDORA-2024-b6801d4514 has been submitted as an update to Fedora 40. https://bodhi.fedoraproject.org/updates/FEDORA-2024-b6801d4514

Comment 8 Fedora Update System 2024-01-26 10:12:13 UTC
FEDORA-2024-b6801d4514 has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 9 Fedora Update System 2024-01-26 11:20:32 UTC
FEDORA-2024-dcec42f261 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2024-dcec42f261

Comment 10 Fedora Update System 2024-01-26 11:20:33 UTC
FEDORA-2024-6c9a7ef1f2 has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2024-6c9a7ef1f2

Comment 11 Fedora Update System 2024-01-27 02:34:37 UTC
FEDORA-2024-6c9a7ef1f2 has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-6c9a7ef1f2`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-6c9a7ef1f2

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2024-01-27 03:05:27 UTC
FEDORA-2024-dcec42f261 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-dcec42f261`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-dcec42f261

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2024-01-29 06:25:39 UTC
FEDORA-2024-6c9a7ef1f2 has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 14 Fedora Update System 2024-02-12 01:51:22 UTC
FEDORA-2024-dcec42f261 (pyproject-rpm-macros-1.12.0-1.fc38) has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.