Bug 2264519

Summary: pyproject-rpm-macros: RFE: link license files into %_datadir/licenses
Product: [Fedora] Fedora Reporter: Petr Menšík <pemensik>
Component: pyproject-rpm-macrosAssignee: Miro Hrončok <mhroncok>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 38CC: maxwell, mhroncok, python-packagers-sig, torsava
Target Milestone: ---Keywords: RFE
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-03-13 13:21:06 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Petr Menšík 2024-02-16 09:59:35 UTC
I were doing my first review on pyproject based package and were demanding a presence of license files.

In bug #2208398 comment #7 I requested adding a license file. Because after quick look there was no %license file in %file list and also no file present in /usr/share/licenses.

Okay, I got corrected on bug #2208398 comment #10 that the file is correctly marked and present in the package always.

But still I think it is installed into very unusual location, where it is not expected. Could be at least symlink created also into %_defaultlicensedir? I think that should be still possible autogenerated, without manual lines in spec file.

It should lower the confusion when someone is trying to find license text where they expect they reside

Reproducible: Always

Steps to Reproduce:
1. fedora-review -b 2208398
2. cd 2208398-*
3. LANG=C.UTF-8 ls -l rpms-unpacked/python3*/usr/share/licenses
Actual Results:  
ls: cannot access 'rpms-unpacked/python3*/usr/share/licenses': No such file or directory


Expected Results:  
lrwxrwxrwx. 1 pemensik pemensik 68 Feb 16 10:56 LICENSE -> ../../lib64/python3.12/site-packages/pyasn-1.6.0b1.dist-info/LICENSE

Comment 1 Miro Hrončok 2024-02-16 10:30:25 UTC
No Fedora packaging guideline demands or even recommends preferring relative paths in %license (i.e. putting licenses to /usr/share/licenses). This was previously "discussed" in https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.org/thread/OHNVT5S4ZPOB56KAYNCWZOC6W2WAV54C/ and clarified in https://pagure.io/packaging-committee/issue/1223

I am sorry, but I see no valid reason why the pyproject macros should create symlinks into %_defaultlicensedir. Your reason seems to be "I expect licenses to be installed in /usr/share/licenses". However, the packaging guidelines explicitly document that this is neither a requirement nor even a preference.


There are also practical problems with what you are proposing:


If the symbolic link is always created in %pyproject_install, packages that don't list it in %files (e.g. because they don't use %files with -f %{pyproject_files}) will fail to build. That would be a *very* breaking change.

If the symbolic link is created by %pyproject_save_files a macro that is only supposed to create a file list will suddenly modify %{buildroot} content. That is a very *weird* behavior. Also, when using %pyproject_save_files packagers are not required to use %files with -f %{pyproject_files} (but they typically do, so this would be a change that might break unusual packages only).

The only way to make this backward compatible (and sane) would be to only create the symbolic link(s) when the packager explicitly uses a specific flag to %pyproject_install or a special macro (e.g. %pyproject_symlink_licenses). But if they need to do that, they might as well just add the %license entry to %files manually instead like:

  %license LICENSE


tl;dr I am reluctant to implement this based on the information you provided. The motivation is not desirable to me and the implementation would be either breaking, weird, or tedious to use.

However, I would be happy to review an idea about the desired behavior that isn't breaking/weird/tedious.

----

Slightly offtopic: %pyproject_save_files only markes the license file with %license if 1) upstream puts it to License-Files, 2) the build backend supports License-Files (e.g. setuptools does).
To ensure a %license is either marked or the build fails, use %pyproject_save_files -l, This is a new feature, see https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/VWBTZT3RJJPFUPYJBBC237KA42KMBI6T/ -- not yet documented in the guidelines, only pyproject-rpm-macros README.

Comment 2 Petr Menšík 2024-02-16 10:44:08 UTC
I never used it myself, but could using hard links help instead? It would still contain only one copy on all systems having even /usr mounted separately. I am not sure how much RPM handles hard links, but if it can use hard link when possible and create duplicate copy when not, it would solve that problem effectively.

But not sure that rpm would detect itself that file is there only once and the second should not be a copy if possible.

Comment 3 Miro Hrončok 2024-02-16 10:52:17 UTC
Hard links in RPM an only possible in one directory, because different directories may be mounted to different file systems. /usr/share and /usr/lib(64) may be mounted elsewhere.

But even if that was not an issue, I don't understand how would hard links solve the problems I pointed out. New files appearing out of nowhere are breaking change (whether they are symbolic links, hard links, copies).

Comment 4 Petr Menšík 2024-02-16 11:31:54 UTC
I have made a little experiment on reviewed spec.

Using %license LICENSE should work after creating a symlink just fine. Sure, it would override LICENSE from dist-info, but quite likely by exactly same file, even if done manually by %license.
It works both with ln -s -r and with ln. At least on the first glance.

diff --git a/python-pyasn.spec b/python-pyasn.spec
index 0ed49da..a38f12a 100644
--- a/python-pyasn.spec
+++ b/python-pyasn.spec
@@ -48,6 +48,8 @@ Summary:        %{summary}
 %install
 %pyproject_install
 %pyproject_save_files %{srcname}
+mkdir -p %{buildroot}%{_defaultlicensedir}/python3-%{srcname}
+ln -s -r %{buildroot}%{python3_sitearch}/%{srcname}-*/LICENSE %{buildroot}%{_defaultlicensedir}/python3-%{srcname}
 
 
 %check
@@ -55,6 +57,7 @@ Summary:        %{summary}
 
 %files -n python3-%{srcname} -f %{pyproject_files}
 %doc README.md BACKLOG.txt
+%license LICENSE
 %{_bindir}/pyasn_util_asnames.py
 %{_bindir}/pyasn_util_convert.py
 %{_bindir}/pyasn_util_download.py

Comment 5 Miro Hrončok 2024-02-16 13:38:52 UTC
So let's consider this addition "happens automagically":

+mkdir -p %{buildroot}%{_defaultlicensedir}/python3-%{srcname}
+ln -s -r %{buildroot}%{python3_sitearch}/%{srcname}-*/LICENSE %{buildroot}%{_defaultlicensedir}/python3-%{srcname}


There is an obvious problem here: pyproject macros don't know the name of the package the LICENSE file will belong to, so they cannot hardcode python3-%{srcname} as you did.
It cannot use %{buildroot}%{_defaultlicensedir}/%{name} because of subpackages.



However, even if we figure out how to make this work, this:

    %license LICENSE

Would need to be added to %{pyproject_files} to avoid breaking all of the packages.

And even if that is done, packages that use %pyproject_install and/or %pyproject_save_files but don't use %files ... -f %{pyproject_files} would require specfile changes.