python-rpm-macros-3-42.fc30 + other Fedoras and ELs ==== Reproducer Consider the following specfile: Name: reproducer_pp Version: 0.0 Release: 1%{?dist} Summary: ... License: MIT %description ... %bcond_with doc %if %{with doc} %package doc Summary: ... BuildArch: noarch %description doc .... %endif %package -n python2-reproducer Summary: ... %{?python_provide:%python_provide python2-reproducer} %description -n python2-reproducer ... %files -n python2-reproducer (end of file) Now build it without doc: $ rpmbuild -ba reproducer_pp.spec --without doc And see the provides: $ rpm -qp --provides ../RPMS/x86_64/python2-reproducer-0.0-1.fc30.x86_64.rpm python-reproducer = 0.0-1.fc30 python-reproducer(x86-64) = 0.0-1.fc30 python2-reproducer = 0.0-1.fc30 python2-reproducer(x86-64) = 0.0-1.fc30 (Notice the archful python-reproducer(x86-64).) Now build with docs: $ rpmbuild -ba reproducer_pp.spec --with doc And see the provides: $ rpm -qp --provides ../RPMS/x86_64/python2-reproducer-0.0-1.fc30.x86_64.rpm python-reproducer = 0.0-1.fc30 python2-reproducer = 0.0-1.fc30 python2-reproducer(x86-64) = 0.0-1.fc30 (Notice the archful python-reproducer(x86-64) is missing.) ==== Problem explained The %python_provide macro checks if buildarch is noarch. If it is not, it adds the archful provide: if (rpm.expand("%{?buildarch}") ~= "noarch") then str = "Provides: python-" .. string.sub(package,9,string.len(package)) .. "%{?_isa} = " .. vr print(rpm.expand(str)) end However the check (%buildarch != noarch) does not check what is intended - if there as an archful main package with multiple subpackages, %buildarch is noarch after the first noarch subpackge is defined. Note that if you reverse the subpackage order in the reproducer, the archful provide is back: ...snip... %package -n python2-reproducer Summary: ... %{?python_provide:%python_provide python2-reproducer} %description -n python2-reproducer ... %if %{with doc} %package doc Summary: ... BuildArch: noarch %description doc .... %endif ...snip... $ rpmbuild -ba reproducer_pp.spec --with doc $ rpm -qp --provides ../RPMS/x86_64/python2-reproducer-0.0-1.fc30.x86_64.rpm python-reproducer = 0.0-1.fc30 python-reproducer(x86-64) = 0.0-1.fc30 python2-reproducer = 0.0-1.fc30 python2-reproducer(x86-64) = 0.0-1.fc30 $ rpmbuild -ba reproducer_pp.spec --without doc $ rpm -qp --provides ../RPMS/x86_64/python2-reproducer-0.0-1.fc30.x86_64.rpm python-reproducer = 0.0-1.fc30 python-reproducer(x86-64) = 0.0-1.fc30 python2-reproducer = 0.0-1.fc30 python2-reproducer(x86-64) = 0.0-1.fc30 ==== Solution? Find a way to check if the "current" package is noarch. I have no idea how to check that. If not possible, document the caveat in the guidelines and recommend defining such noarch packages after all archful packages that use %python_provide. https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_the_python_provide_macro
So first off, I think there's at least a good chance that this would be considered an upstream RPM bug. I filed https://github.com/rpm-software-management/rpm/issues/689 If that were fixed then perhaps the current method could be made to work. But since RHEL is involved, we can't assume that this is fixable. So the only ways out I can think of are: * Suggest reordering subpackage declarations to put %noarch at the end * Add an option to the macro to force archfulness (if that's possible) * Add a new %python_provide_arch macro which forces archfulness
So I dug a bit and found that the situation is a bit different than I thought. Stick this in /usr/lib/rpm/macros.d/macros.test: %foofoo() %{lua: lba = rpm.expand("%{buildarch}") uba = rpm.expand("%{BUILDARCH}") name = rpm.expand("%name") prefix = rpm.expand("%1") rpm.expand("%{warn:" .. prefix .. ":Name is " .. name .. "}") rpm.expand("%{warn:" .. prefix .. ":buildarch is " .. lba .. "}") rpm.expand("%{warn:" .. prefix .. ":BUILDARCH is " .. uba .. "}") } And stick "%foofoo XXX" or whatever throughout your spec, changing XXX each time. Basically %buildarch isn't defined until RPM sees BuildArch: noarch somewhere in the spec. Then it's set to noarch. So it's... pretty much entirely useless for what the macro is trying to do (and pretty much useless in general).
However those 2 examples bahave the same: ------------------------------------------------------ Name: reproducer_pp Version: 0.0 Release: 1%{?dist} Summary: ... License: MIT %{?python_provide:%python_provide python2-reproducer} BuildArch: noarch %description ... %files ------------------------------------------------------ Name: reproducer_pp Version: 0.0 Release: 1%{?dist} Summary: ... License: MIT BuildArch: noarch %{?python_provide:%python_provide python2-reproducer} %description ... %files ------------------------------------------------------
So based on the upstream fix [1] it seems that this will now stop working altogether (all packages will be treated as arched, possibly with different provides based on the actual arch of the builder). [1] https://github.com/rpm-software-management/rpm/commit/7ca03925d88d27266a4f8bec7a35e8c3b1117279
In Rawhide, let's change the macro to not add the arched Provides. Document that you need to add a second %{python_provide} macro with %{_isa} to get the old behavior.
I wonder if it possible to somehow leverage the automatic dependency generator to do this instead of adding macros. Unfortunately dep generators only have access to a list of filenames and whatever options you pass in %__XXX_provides_opts. I'm not sure that's enough. Dep generators may seem like black magic but they aren't really that difficult. If you think a file list is enough then I could look into hacking on it a bit.
We already generate provides based on package contents. %python_provide is mostly for provides based on package names. Honestly, I don't think the arched provides such as python-reproducer(x86-64) are any good, the provide is mostly for users, so they can do: dnf install python-foo ...and still get something. They are forbidden to be used in Fedora as (Build)Requires, and I have (probably) never seen %{_isa} required Python package anyway. I'm with Petr here: We make sure the behavior is not broken (such as sticking %{_isa} to all packages incl. noarch) and we document that the macro only adds arch-agnostic provides, with a workaround for those who would be sad about that. As packaging guidelines go, they remain intact: packages SHOULD use %python_provide with package name.
I almost reassigned bug 1720139 to python-rpm-macros for discussion and awareness but realized this bug already exists, and was even pointed to in the upstream ticket, which I reopened because of bug 1720139. I must be going blind... I'm not unsympathetic to the cause, even if the upstream fix of removing the whole thing might seem that way. It's more about trying to find a sane, reliable way to make such data available to packages. Anything in the spec preamble (global or sub-package level) is inherently order-dependent, for a proper solution there would have to be some sort of "post slot" which takes place after the preamble has been parsed. Dependency generators are one such slot, but they're very file-centric at the moment so doesn't lend itself too well for this purpose. Ideas welcome. OTOH if you think the arched provides are not worth the trouble then now would be the time to drop it to allow rpm to drop the buildarch macro for real.
It's not actually the dependency on ordering that causes a problem, and as I understand things RPM's spec parsing works in a single pass with macro expansion happening at parse time. And that's fine. What was confusing here (which caused me to open the upstream ticket about this) is that RPM doesn't consistently track in "real-time" the state of the parser in a way that macros can access. So while it sort of looked like we could rely on %buildarch to tell us something about the architecture of the package/subpackage that is currently being defined (assuming you put the macro after the BuildArch: line), it simply doesn't work that way. I didn't write the macro originally but I was fooled by the mere existence of %buildarch and the fact that it did get defined to 'noarch' as soon as a single BuildArch: noarch line was seen. It does seem like it works, and indeed it sort of does if you put all of your noarch package definitions at the end. But it seems that's merely by accident. I think in this case that Miro is right and that the arch-specific output that %python_provide was trying (and failing) to output isn't all that useful, but I do think in general that it would be nice to be able to get at such information. (Key things like the summary and description would be really nice to be able to access. At one point to get at %description I wrote a parser in Lua and reparsed the current spec from /proc/self/fd/3.) It would still be quite useful even if it's only in a Lua table and not exposed as traditional RPM macros. (There's precedent for that in the way patches and sources work, I think.)
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/22
The arched part has been removed. Should we update the guidelines to mention the workaround if people need it back, or not? I think it is very unlikely that somebody was counting on this.
Right, that does seem unnecessary -- though we will want to watch the mass rebuild for the missing requirements.
The mass rebuild happened and the Python 3.8 rebuild happened, I've noticed no problems.