Bug 1394853 - [RFE] support ExclusiveArch tag in binary (sub)packages, too
Summary: [RFE] support ExclusiveArch tag in binary (sub)packages, too
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-14 15:13 UTC by Pavel Raiskup
Modified: 2019-10-15 08:41 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-15 08:41:02 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1298668 0 unspecified CLOSED Allow noarch builds with ExclusiveArch not containing noarch 2021-02-22 00:41:40 UTC

Internal Links: 1298668

Description Pavel Raiskup 2016-11-14 15:13:31 UTC
So far, ExclusiveArch tag seems to be baked into source RPM only.  That
however complicates rel-eng decisions about "what architecture is
particular noarch _binary_ package compatible with".

Example of the problem:

  I have a spec file foo.spec, which generates three 'noarch' packages
  'foo' and 'foo-libs' and 'foo-features'.  Package 'foo-features' is told
  to be 'x86_64' only by 'ExclusiveArch: x86_64' (for whatever reasons,
  let's say that there's no point to install 'foo-features' anywhere
  else).

Pungi (rel-eng tool) goes (at least I think so) through the list of
"binary" packages one-by-one, and because the 'foo' package is 'noarch',
it looks at corresponding src.rpm for ExclusiveArch flag [1].

The ultimate problem is that 'foo' is 'noarch' and compatible with all
architectures Fedora supports, but it is also excluded... because, at
least I think) there's only one ExclusiveArch tag per Source RPM.

The solution could be:

  1. bake the ExclusiveArch into binary (at least noarch) RPM too
  2. invent another RPM tag to ease rel-eng's job, then please fix the
     $Summary field.

[1] https://pagure.io/pungi-fedora/issue/87

Comment 1 Pavel Raiskup 2016-11-14 15:16:21 UTC
Somehow related bug added to 'See also' (people try to workaround that bug by
putting 'noarch' into ExclusiveArch tag, which looks like it effectively makes
the package compatible with every architecture from pungi (Rel Eng) POV).

Comment 2 Tomasz Kłoczko 2019-03-10 11:02:18 UTC
There is nothing to do here to extend rpm.
All what needs to be done in the spec file is:

%ifarch x86_64
%files foo-features
..
..
%endif

Comment 3 Pavel Raiskup 2019-03-11 06:55:12 UTC
Let's consider reasons why people do 'ExclusiveArch: noarch', and apply
those reasons on sub-packages...  so no.

Your solution doesn't work for 'BuildArch: noarch' packages, where %ifarch
x86_64 is always false.  Also, it wouldn't be nothing than ugly hack to
provide empty subpackages because the build system sucks...  I rather
decided to not ship those subpackages at all.

The goal of this bug-report once more
=====================================

I need a mechanism to mark specific (noarch) sub-package to be
only available on certain subset of architectures;  so pungi only
adds them to correct arch-specific yum repositories.

Comment 4 Pavel Raiskup 2019-03-11 07:03:45 UTC
Sorry, this bug is more rel-eng oriented and explained for rel-engs I believe.

The example from
https://src.fedoraproject.org/rpms/vim-syntastic/c/8160444cb71b9901471ea97bd00a1a7544f355eb
might better explain what is going on.  If I've had a sub-package that is
**not** installable everywhere (say only on s390x), the whole _package_ (not only
sub-package) would be dropped from fedora after certain period of time.

So I decided to protect the whole package from removal by dropping the
guilty sub-packages (instead of playing with %filter_requires, which is the
only working - but still ugly - work-around I guess...).

Comment 5 Tomasz Kłoczko 2019-03-11 07:34:14 UTC
Most of the Fedora packages with "BuildArch: noarch" and "ExclusiveArch: <other_archs> noarch" are nodejs specs which are producing only one package. Typically it is like this:

BuildArch:      noarch
ExclusiveArch:  %{nodejs_arches} noarch

Where:
$ rpm -E %nodejs_arches
i386 i486 i586 i686 pentium3 pentium4 athlon geode x86_64 armv3l armv4b armv4l armv4tl armv5tl armv5tel armv5tejl armv6l armv6hl armv7l armv7hl armv7hnl aarch64 ppc64 ppc64p7 ppc64le s390x

So those packages have listed in ExclusiveArch ALL possible archs.
It does not make any sense and in all those cases in specs should be only "BuildArch: noarch".
In other words this bug is imaginary .. not real.
The same is with vim-syntastic.spec. All subpackages generated out of this spec are noarch because all those subpackages provides only exact language syntax support.

Comment 6 Pavel Raiskup 2019-03-11 07:43:02 UTC
(In reply to Tomasz Kłoczko from comment #5)
> In other words this bug is imaginary .. not real.

No, it is not .. read the example and commits in vim-syntastic, or ping
me on irc;  I'm afraid I don't know how to follow up here and I don't want
to make the bug even harder to understand.

> The same is with vim-syntastic.spec. All subpackages generated out of this
> spec are noarch because all those subpackages provides only exact language
> syntax support.

That is right, and that's why I can not make it non-noarch.

But while majority of the subpackages is compatible and installable
on all our architectures - some are not.  And I don't want to remove the whole)
vim-syntastic package from aarch64 only because "clisp" is not being available
on aarch64 or so.  Please submit a PR and link it here if you think that
there is a solution...

Comment 7 Tomasz Kłoczko 2019-03-11 09:05:04 UTC
> No, it is not .. read the example and commits in vim-syntastic, or ping me on irc;  I'm afraid I don't know how to follow up here and I don't want to make the bug even harder to understand.

In %changelog is possible to find entry:
> drop two subpackages which are not installable on aarch64/ppc64le due to
> missing dependencies, till we have fixed rel-eng scripts or we ratify

So it has nothing to do with this package but with some other packages which are missing on exact build archs when vim-syntastic is build.
Let me ask: what is the sense building noarch packages on multiple exact arch builders?
Here is is the hook on which hangs whole assumption that something like vim-syntastic is something arch specific.

Just port that missing package to missing arch and nothing in rpm needs to be changed.

If you still have some doubts just point on single file in vim-syntastic which is arch specific.

Comment 8 Pavel Raiskup 2019-03-11 09:25:28 UTC
(In reply to Tomasz Kłoczko from comment #7)
> > No, it is not .. read the example and commits in vim-syntastic, or ping me
> > on irc;  I'm afraid I don't know how to follow up here and I don't want to
> > make the bug even harder to understand.
> 
> In %changelog is possible to find entry:
> > drop two subpackages which are not installable on aarch64/ppc64le due to
> > missing dependencies, till we have fixed rel-eng scripts or we ratify
> 
> So it has nothing to do with this package but with some other packages which
> are missing on exact build archs when vim-syntastic is build.

Yes.  But for some reason I'm not able to explain, or you are not able to
understand the problem.  Please could you try to fix 'vim-syntastic-lisp' so
it is shipped as 'noarch', and is available only on architectures where it can
be installed?  That will explain everything.  I'm afraid we are making this
bug report TL;DR :-(

> Let me ask: what is the sense building noarch packages on multiple exact
> arch builders?

I don't know why you think I want this.

> Here is is the hook on which hangs whole assumption that something like
> vim-syntastic is something arch specific.

People told me that I should make vim-syntastic arch-specific to work-around
broken pungi, but I refused to do so.

> Just port that missing package to missing arch and nothing in rpm needs to
> be changed.

Please, how do you expect that release-engineering team (pungi) should detect
that particular (noarch) sub-package isn't to be shipped on aarch64?

As I said, this bug was meant to be kick-off for release engineers to specify
"how" rpm should implement this.  But it's 3 years, and no reaction.

> If you still have some doubts just point on single file in vim-syntastic
> which is arch specific.

Please contact me interactively, I'll explain you ... and I'll follow up here
once I'll understand what you don't understand.  Thanks.

Comment 9 Panu Matilainen 2019-03-11 10:35:02 UTC
Pavel, no need to further argue or explain, the problem of noarch packages with arch-specific dependencies is old as ages and well known, just one that tries hard to be forgotten ;) In part because this is not really rpm's problem.

For the clisp/nodejs kind of case, having innocent noarch binary packages carry somebody elses incompatibility-information seems counter-productive to me, because then if the previously missing dependency appears on the previously missing architecture, you'd need to adjust and rebuild all those noarch packages for no good reason at all.

That is a compose-side problem, and basically rel-eng tools should just skip packages whose dependencies can't be met in the target (arch) repo, but of course it's not that simple either. I'm just not convinced it's necessary rpm where something needs to be implemented to support this.

Comment 10 Pavel Raiskup 2019-03-11 10:45:52 UTC
> That is a compose-side problem, and basically rel-eng tools should just
> skip packages whose dependencies can't be met in the target (arch) repo,

I'm afraid that average compose would loose tens of % packages, because of
packaging issues if we did this.  I'd be afraid of performance issues,
too.

> but of course it's not that simple either. I'm just not convinced it's
> necessary rpm where something needs to be implemented to support this.

I was thinking about something really explicit - so I as a packager can
claim "this" package is **not** to be shipped on ArchFoo.  Rather then
doing some analysis I like determinism.

Well, we don't really have to have this in RPM standard;  we might well
have something like '%arches_unsupported ppc64' macro, which would
generate 'Provides: arch_not_supported(ppc64)' (or so).  Would that make
more sense?

Comment 11 Panu Matilainen 2019-03-11 11:52:10 UTC
Like said, I know it's not as simple as just skipping packages based on dependencies.
I just don't think that the binary noarch packages is the right place for that information. A compose-side tool could just as well look at a plain-text file in dist-git, or something like that.

Comment 12 Panu Matilainen 2019-03-11 12:12:01 UTC
Oh in the above I'm talking about the "not usable on arch X because a dependency Y is missing" case. There is, at least in theory, the other case where a package that's technically noarch just doesn't make any sense on some architectures although it might even run - say, a S390-specific system management shell-script. Easily worked around by making it arch-specific of course, but whether that's the "right thing", dunno.

Comment 13 Pavel Raiskup 2019-03-11 12:16:06 UTC
> A compose-side tool could just as well look at a
> plain-text file in dist-git, or something like that.

It could, though ... would it be wiser to spread the package metadata to
another place?  (I guess that by dist-git you don't mean rpms/<PACKAGE>.git
repo, because that would mean that all packages' dist-git repos need to be
cloned for each compose, and checked out to corresponding branches, etc.).

Comment 14 Tomasz Kłoczko 2019-03-11 13:11:19 UTC
(In reply to Panu Matilainen from comment #12)
> Oh in the above I'm talking about the "not usable on arch X because a
> dependency Y is missing" case. There is, at least in theory, the other case
> where a package that's technically noarch just doesn't make any sense on
> some architectures although it might even run - say, a S390-specific system
> management shell-script. Easily worked around by making it arch-specific of
> course, but whether that's the "right thing", dunno.

If you will look on the issue from a bit bit different angle you may find it that this is not the rpm issue but build infrastructure or still not well ported set f shrinking packages.
Please do not try to make rpm more complicated tool that it is now.
As well something like rpm RFE should be not discussed here but on rpm-devel list (however I'm sure that rpm maintainer will refuse to do anything because it is nothing to do with rpm).

Comment 15 Pavel Raiskup 2019-03-11 14:35:19 UTC
RPM describes packages, packagers use it enhance the shipped software by
some metadata.  And release-engineers need some more metadata in this
case, the question is where those should live.

Note that solution without additional RPM tag will be just another
distro-specific hack, which means that each RPM distribution will have to
deal with this problem on it's own.  I'm not saying that we can not go out
on our own, I'm just saying that it is not that clear what should be done
(and there are certainly cons).  There _would_ be some benefits in
standardizing this.

Tomasz, claiming it is "complicated" is POV and probably FUD.  I would really
rather hear something constructive than "do nothing", this "is not a problem",
go and fix your package, go and fix "clisp stack", "D stack", don't ship those
packages if they don't work everywhere, make the package arch-specific, ...

Comment 16 Tomasz Kłoczko 2019-03-11 15:57:59 UTC
(In reply to Pavel Raiskup from comment #15)
[..]
> Tomasz, claiming it is "complicated" is POV and probably FUD.  I would really
> rather hear something constructive than "do nothing", this "is not a
> problem",
> go and fix your package, go and fix "clisp stack", "D stack", don't ship
> those packages if they don't work everywhere, make the package arch-specific, ...

I'm not rpm code maintainer. You are discussing topic with wrong person.
I have my own opinion and I would be very sensitive in case packaging anything which has portability issues. Personally I would recommend to fix potability first (because portability it may be only tip of the iceberg) and your problem will automatically will be solved.
You cannot expect that ot will be possible to package every possible crap software available in this universe because the set of problems which such crap software may create will be infinite (continuum).
This is like expecting that some compiler will be able to compile every *.c file (whatever it will contain).
No. Firs solve some basic problems.
(but again .. this is only my humble private opinion that this should be sorted out that way)

PS. I would really suggest really close this ticket and move to rpm-devel mailing list because this issue has nothing to do wit rpm provided ads package in exact distribution.

Comment 17 Panu Matilainen 2019-03-12 06:20:31 UTC
> I'm not rpm code maintainer. You are discussing topic with wrong person.

Then how about you just shut up instead of telling others what to do on a matter that's self-admittedly is none of your business?

Comment 18 Panu Matilainen 2019-10-15 08:41:02 UTC
This recently came up at upstream at https://github.com/rpm-software-management/rpm/issues/865, to quote myself:

I think this is a compose problem, not rpm problem: it'd be wrong for rpm to encode some arch-specific knowledge into that noarch package, when that arch-limitation could go away at any time (by that extra package becoming available on that arch). Another line of thought is what dnf did initially and simply hide packages with broken dependencies - arguably that's what a repo compose should do.

Adding arch-limitations to noarch packages in rpm is not going to happen, so WONTFIX for this particular request.


Note You need to log in before you can comment on or make changes to this bug.