This bug has been migrated to another issue tracking site. It has been closed here and may no longer be being monitored.

If you would like to get updates for this issue, or to participate in it, you may do so at Red Hat Issue Tracker .
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2102548 - glibc-gconv-extra-2.28-206.el8.x86_64.rpm missing from AppStream
Summary: glibc-gconv-extra-2.28-206.el8.x86_64.rpm missing from AppStream
Keywords:
Status: CLOSED MIGRATED
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: distribution
Version: CentOS Stream
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
: ---
Assignee: Brian Stinson
QA Contact: Release Test Team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-06-30 07:56 UTC by Wong Hoi Sing Edison
Modified: 2023-09-14 12:27 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-09-14 12:25:39 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
nothing provides glibc(x86-64) = 2.28-203.el8 needed by glibc-gconv-extra (86.55 KB, image/png)
2022-06-30 07:56 UTC, Wong Hoi Sing Edison
no flags Details
https://build.opensuse.org/projects/home:alvistack/meta (266.55 KB, image/png)
2022-06-30 12:56 UTC, Wong Hoi Sing Edison
no flags Details
nothing provides libxml2(x86-64) = 2.9.7-11.el8 needed by libxml2-devel, (got version 2.9.7-14.el8 provided by libxml2) (136.00 KB, image/png)
2022-07-02 00:30 UTC, Wong Hoi Sing Edison
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 2103293 0 unspecified CLOSED libxml2-devel-2.9.7-14.el8.x86_64.rpm missing from AppStream 2022-07-05 17:36:07 UTC
Red Hat Issue Tracker   RHEL-3756 0 None Migrated None 2023-09-14 12:25:34 UTC
Red Hat Issue Tracker RHELPLAN-126703 0 None None None 2022-06-30 08:06:11 UTC

Internal Links: 2103293

Description Wong Hoi Sing Edison 2022-06-30 07:56:30 UTC
Created attachment 1893558 [details]
nothing provides glibc(x86-64) = 2.28-203.el8 needed by glibc-gconv-extra

Description of problem:

From https://linux.cc.iitk.ac.in/mirror/centos/8/AppStream/x86_64/os/Packages/ we have https://linux.cc.iitk.ac.in/mirror/centos/8/AppStream/x86_64/os/Packages/glibc-gconv-extra-2.28-206.el8.i686.rpm, but missing https://linux.cc.iitk.ac.in/mirror/centos/8/AppStream/x86_64/os/Packages/glibc-gconv-extra-2.28-206.el8.x86_64.rpm; it is only available from BaseOS with https://linux.cc.iitk.ac.in/mirror/centos/8/BaseOS/x86_64/os/Packages/glibc-gconv-extra-2.28-206.el8.x86_64.rpm.

This lead to https://build.opensuse.org report error message as below (in case we setup AppStream as higher priority than BaseOS, which usually working well):

> nothing provides glibc(x86-64) = 2.28-203.el8 needed by glibc-gconv-extra, (got version 2.28-206.el8 provided by glibc), nothing provides glibc-common = 2.28-203.el8 needed by glibc-gconv-extra, (got version 2.28-206.el8)

Currently all of my CentOS-8:Stream builds at OBS are failing with missing package, see https://build.opensuse.org/project/monitor/home:alvistack?arch_x86_64=1&defaults=0&repo_CentOS_8_Stream=1&unresolvable=1

Comment 1 Florian Weimer 2022-06-30 08:43:35 UTC
(In reply to Wong Hoi Sing Edison from comment #0)
> Created attachment 1893558 [details]
> nothing provides glibc(x86-64) = 2.28-203.el8 needed by glibc-gconv-extra
> 
> Description of problem:
> 
> From
> https://linux.cc.iitk.ac.in/mirror/centos/8/AppStream/x86_64/os/Packages/ we
> have
> https://linux.cc.iitk.ac.in/mirror/centos/8/AppStream/x86_64/os/Packages/
> glibc-gconv-extra-2.28-206.el8.i686.rpm, but missing
> https://linux.cc.iitk.ac.in/mirror/centos/8/AppStream/x86_64/os/Packages/
> glibc-gconv-extra-2.28-206.el8.x86_64.rpm; it is only available from BaseOS
> with
> https://linux.cc.iitk.ac.in/mirror/centos/8/BaseOS/x86_64/os/Packages/glibc-
> gconv-extra-2.28-206.el8.x86_64.rpm.
> 
> This lead to https://build.opensuse.org report error message as below (in
> case we setup AppStream as higher priority than BaseOS, which usually
> working well):
> 
> > nothing provides glibc(x86-64) = 2.28-203.el8 needed by glibc-gconv-extra, (got version 2.28-206.el8 provided by glibc), nothing provides glibc-common = 2.28-203.el8 needed by glibc-gconv-extra, (got version 2.28-206.el8)

This really shouldn't cause any problems for DNF. I suspect the error is coming from something else. Could you share a longer log message with the full error message? Thanks.

Comment 2 Wong Hoi Sing Edison 2022-06-30 12:56:18 UTC
Created attachment 1893658 [details]
https://build.opensuse.org/projects/home:alvistack/meta

> This really shouldn't cause any problems for DNF. I suspect the error is coming from something else. Could you share a longer log message with the full error message? Thanks.

Agree that may not affect DNF, because we will load in ALL repos with whatever order, then calculate the highest available and suitable version, pick it up and install correctly. So even if AppStream unexpectedly missing glibc-gconv-extra-2.28-206.el8.x86_64.rpm, we will pick it up from BaseOS so don't break anything.

But sorry that with OBS limitation as mentioned from https://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks#Adding_multiple_repositories_to_a_project:

> The order of the included repositories is important: The BuildService will try to use a package from the first repository that includes the package, even if the version doesn't match.

As screenshot attached, since my project setup with AppStream > BaseOS, in case AppStream already have glibc-gconv-extra with legacy version 2.28-203 only, OBS will pick it up because 2.28-206 is now missing, and will not fallback to BaseOS for searching available package.

IMHO, since we have all expected legacy version for glibc-gconv-extra with both i686 and x86_64 but nwo only missing for 2.28-206.el8.x86_64, it seems to be a simple careless bug for me which could be fixed easily?

Comment 3 Florian Weimer 2022-06-30 13:28:53 UTC
Interesting. CentOS Koji publishes buildroot repositories here:

https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/

These repositories are not split along the BaseOS/AppStream lines and should therefore work with OBS. The package versions should match what is available in the compose (but the compose may not have hit the CentOS mirror network yet).

Can you switch to these repositories as a workaround? Thanks.

Comment 4 Wong Hoi Sing Edison 2022-06-30 14:14:47 UTC
> Can you switch to these repositories as a workaround? Thanks.

First of all I am not an OBS admin but a normal user, so I have no right for switching the repo...

Moreover, from https://build.opensuse.org/projects/CentOS:CentOS-8:Stream/meta, their AppStream repo is now pointing to http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/, which also missing http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/glibc-gconv-extra-2.28-206.el8.x86_64.rpm...

Finally, I always checking package availability with https://pkgs.org/search/?q=glibc-gconv-extra, but also show that missing glibc-gconv-extra-2.28-206.el8.x86_64.rpm from CentOS 8 Stream officially...

Comment 5 Wong Hoi Sing Edison 2022-07-01 09:52:54 UTC
@fweimer I give different workaround with my own https://build.opensuse.org/projects/home:alvistack/meta, but:

* If AppStream > BaseOS (the style which working fine for last year until this bug report), only glibc-gconv-extra hit the version miss match issue

* If BaseOS > AppStream, now libxml2-devel get version miss match with libxml2...

* If remove AppStream but only keep BaseOS, now multiple packages get version miss match, too...

Both behavior happened with CentOS 8 Stream and CentOS 9 Stream, which clearly show that CentOS Stream now a day heavily depends on AppStream repo (as a replacement for CentOS 7 update repo).

Understand that CentOS Koji didn't split BaseOS/AppStream lines, but package is now disappeared from all mirror network, which seems like some unexpected pipeline error?

Comment 6 Brian Stinson 2022-07-01 15:22:14 UTC
glibc-gconv-extra hasn't disappeared though, it moved to BaseOS:
http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/glibc-gconv-extra-2.28-206.el8.x86_64.rpm

I also notice that the i686 package stayed in AppStream:
http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/glibc-gconv-extra-2.28-206.el8.i686.rpm

@fweimer do we need to address that?

Comment 7 Wong Hoi Sing Edison 2022-07-01 15:37:37 UTC
From https://pkgs.org/search/?q=glibc and https://pkgs.org/search/?q=glibc-gconv-extra, showing that:

* For BaseOS, originally glibc-gconv-extra <= 2.28-180 (but now, unexpectedly 2.28-206 push to here)

* For AppStream, originally glibc-gconv-extra >= 2.28-181 (but now, unexpectedly 2.28-206 is missing)

Since now glibc and glibc-common from BaseOS already go to 2.28-206, if OBS repo order config as AppStream > BaseOS, normally it should pick up glibc-gconv-extra = 2.28-206 from AppStream, and so no error happened.

IMHO, the move of glibc-gconv-extra = 2.28-206 from BaseOS to AppStream shouldn't be a feature but a bug?

Also see https://lists.opensuse.org/archives/list/buildservice@lists.opensuse.org/thread/UJ4ZNOKJ2AN3NUUUYUH4M4K2FSZUI3D4/

Comment 8 Florian Weimer 2022-07-01 18:57:19 UTC
(In reply to Wong Hoi Sing Edison from comment #5)
> * If BaseOS > AppStream, now libxml2-devel get version miss match with
> libxml2...

I asked internally, and prioritizing BaseOS over AppStream is supposed to work because we never remove packages from BaseOS and add them to AppStream. This means that the version in BaseOS is always newer.

If that's not true for libxml2-devel, that is a separate bug, but I do not immediately see that this is the case.

> Both behavior happened with CentOS 8 Stream and CentOS 9 Stream, which
> clearly show that CentOS Stream now a day heavily depends on AppStream repo
> (as a replacement for CentOS 7 update repo).

AppStream has always been an integral part.  I think the difference relative to 7 is that CentOS now follows the Red Hat Enterprise Linux repository structure more closely, which leads to fewer surprises for those of us who live in both worls.

> Understand that CentOS Koji didn't split BaseOS/AppStream lines, but package
> is now disappeared from all mirror network, which seems like some unexpected
> pipeline error?

Which package has disappeared? I don't think CentOS removes packages.

Comment 9 Florian Weimer 2022-07-01 19:10:52 UTC
(In reply to Brian Stinson from comment #6)
> glibc-gconv-extra hasn't disappeared though, it moved to BaseOS:
> http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/glibc-
> gconv-extra-2.28-206.el8.x86_64.rpm
> 
> I also notice that the i686 package stayed in AppStream:
> http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/glibc-
> gconv-extra-2.28-206.el8.i686.rpm
> 
> @fweimer do we need to address that?

The packages should be in BaseOS for all architectures. This is what we requested for Red Hat Enterprise Linux 8 & 9, and the repositories seem to match.

Comment 10 Florian Weimer 2022-07-01 19:18:17 UTC
(In reply to Wong Hoi Sing Edison from comment #7)
> Also see
> https://lists.opensuse.org/archives/list/buildservice@lists.opensuse.org/
> thread/UJ4ZNOKJ2AN3NUUUYUH4M4K2FSZUI3D4/

I commented on that thread as well, thanks.

Brian, do we need to keep this bug open to track the CentOS change, or can we close it? Thanks.

Comment 11 Brian Stinson 2022-07-01 19:53:16 UTC
If there are no further issues, let's reassign this to the distribution component and I'll take assignment. We'll complete the move to BaseOS in any case.

Comment 12 Wong Hoi Sing Edison 2022-07-02 00:30:44 UTC
Created attachment 1894009 [details]
nothing provides libxml2(x86-64) = 2.9.7-11.el8 needed by libxml2-devel, (got version 2.9.7-14.el8 provided by libxml2)

For your reference if I move BaseOS > AppStream (which means if they both have a specific package with different version, OBS will ALWAYS install it and its dependency from BaseOS ONLY), now most packages could compile successfully.

BTW, still 15 packages failed due to "nothing provides libxml2(x86-64) = 2.9.7-11.el8 needed by libxml2-devel, (got version 2.9.7-14.el8 provided by libxml2)", which I am going to report as another ticket.

Moreover, if "glibc-gconv-extra should move from AppStream to BaseOS" as official decision, could we move ALL LEGACY AppStream glibc-gconv-extra packages to BaseOS, too?

Comment 13 Christian 2022-11-30 22:23:04 UTC
CentOS 8 Stream builds are still breaking on OBS for us for the above reason. This is pretty frustrating because we don't know who can fix this — the OBS folks or CentOS, or us? Has anyone identified a workaround for folks who just want to package code for a bunch of platforms on OBS?

Fwiw, here are related threads:

https://lists.opensuse.org/archives/list/buildservice@lists.opensuse.org/thread/2CWV33W5XZNKT6WJRDBUPAANGWQHDH5H/
https://lists.opensuse.org/archives/list/buildservice@lists.opensuse.org/thread/A2T7CATUGIDFCO7JFTNKWFTFBPTY3I7Z/

And the gist of the matter on OBS:

https://build.opensuse.org/project/monitor/security:zeek?arch_x86_64=1&defaults=0&repo_CentOS_8_Stream=1&unresolvable=1

Thanks!

Comment 14 Neal Gompa 2022-12-03 17:18:08 UTC
(In reply to Florian Weimer from comment #3)
> Interesting. CentOS Koji publishes buildroot repositories here:
> 
> https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/
> 
> These repositories are not split along the BaseOS/AppStream lines and should
> therefore work with OBS. The package versions should match what is available
> in the compose (but the compose may not have hit the CentOS mirror network
> yet).
> 
> Can you switch to these repositories as a workaround? Thanks.

It unlikely that the openSUSE Build Service admins would use that, since neither the RPMs nor the repodata are signed. OBS relies on at least repodata signing to verify the integrity of build inputs.

Comment 15 RHEL Program Management 2023-09-14 12:23:38 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 16 RHEL Program Management 2023-09-14 12:25:39 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.


Note You need to log in before you can comment on or make changes to this bug.