Bug 2138460 - Unable to update ARM AMIs on AWS [NEEDINFO]
Summary: Unable to update ARM AMIs on AWS
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Update Infrastructure for Cloud Providers
Classification: Red Hat
Component: Operations
Version: 4.0.1
Hardware: aarch64
OS: Linux
unspecified
urgent
Target Milestone: ---
: ---
Assignee: RHUI Bug List
QA Contact: Radek Bíba
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-10-28 22:49 UTC by peninm
Modified: 2023-06-28 13:11 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-06-28 13:11:10 UTC
Target Upstream Version:
Embargoed:
peninm: needinfo? (tpfromme)


Attachments (Terms of Use)

Description peninm 2022-10-28 22:49:26 UTC
Description of problem:
Unable to update ARM AMIs on AWS due to an issue with NetworkManager.


Version-Release number of selected component (if applicable):
package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64 requires NetworkManager = 1:1.36.0-7.el8_6

How reproducible:
Happens every time

Steps to Reproduce:
1. Launch ARM (RHEL-8.6.0_HVM-20220503-arm64-2-Hourly2-GP2) version of RHEL 8 on AWS
2. Run `sudo dnf update`
3. Observe failure

Actual results:
Failure to update

Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered with an entitlement server. You can use subscription-manager to register.

Last metadata expiration check: 0:03:53 ago on Fri 28 Oct 2022 10:42:35 PM UTC.
Error:
 Problem 1: package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64 requires NetworkManager = 1:1.36.0-7.el8_6, but none of the providers can be installed
  - cannot install both NetworkManager-1:1.36.0-9.el8_6.aarch64 and NetworkManager-1:1.36.0-7.el8_6.aarch64
  - cannot install the best update candidate for package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64
  - cannot install the best update candidate for package NetworkManager-1:1.36.0-4.el8.aarch64
 Problem 2: problem with installed package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64
  - package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64 requires NetworkManager-libnm(aarch-64) = 1:1.36.0-4.el8, but none of the providers can be installed
  - package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64 requires NetworkManager-libnm(aarch-64) = 1:1.36.0-7.el8_6, but none of the providers can be installed
  - cannot install both NetworkManager-libnm-1:1.36.0-9.el8_6.aarch64 and NetworkManager-libnm-1:1.36.0-4.el8.aarch64
  - cannot install both NetworkManager-libnm-1:1.36.0-9.el8_6.aarch64 and NetworkManager-libnm-1:1.36.0-7.el8_6.aarch64
  - cannot install the best update candidate for package NetworkManager-libnm-1:1.36.0-4.el8.aarch64
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Expected results:
AMI installs latest update

Additional info:

Comment 1 Gregg Tanzillo 2022-10-29 04:18:59 UTC
I was able to recreate the issue, following the steps in the BZ (see output below). However, I don’t know if it’s a RHUi issue or possibly a problem with the package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64, itself or the RHUI client on that image or an entitlement issue. 

I've emailed RHUI engineering asking for their assistance.

Comment 2 Gregg Tanzillo 2022-10-29 04:21:02 UTC
Here's my output:

[ec2-user@ip-172-32-32-42 ~]$ sudo dnf update
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered with an entitlement server. You can use subscription-manager to register.

Red Hat Enterprise Linux 8 for aarch64 - AppStream from RHUI (RPMs)                                                                                                                                                                        33 MB/s |  39 MB     00:01
Red Hat Enterprise Linux 8 for aarch64 - BaseOS from RHUI (RPMs)                                                                                                                                                                           41 MB/s |  42 MB     00:01
Red Hat Ansible Engine 2 for RHEL 8 (RPMs) from RHUI                                                                                                                                                                                      9.7 MB/s | 2.5 MB     00:00
RHUI Client Configuration Server 8                                                                                                                                                                                                         29 kB/s | 2.3 kB     00:00
Error:
 Problem 1: package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64 requires NetworkManager = 1:1.36.0-7.el8_6, but none of the providers can be installed
  - cannot install both NetworkManager-1:1.36.0-9.el8_6.aarch64 and NetworkManager-1:1.36.0-7.el8_6.aarch64
  - cannot install the best update candidate for package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64
  - cannot install the best update candidate for package NetworkManager-1:1.36.0-4.el8.aarch64
 Problem 2: problem with installed package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64
  - package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64 requires NetworkManager-libnm(aarch-64) = 1:1.36.0-4.el8, but none of the providers can be installed
  - package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64 requires NetworkManager-libnm(aarch-64) = 1:1.36.0-7.el8_6, but none of the providers can be installed
  - cannot install both NetworkManager-libnm-1:1.36.0-9.el8_6.aarch64 and NetworkManager-libnm-1:1.36.0-4.el8.aarch64
  - cannot install both NetworkManager-libnm-1:1.36.0-9.el8_6.aarch64 and NetworkManager-libnm-1:1.36.0-7.el8_6.aarch64
  - cannot install the best update candidate for package NetworkManager-libnm-1:1.36.0-4.el8.aarch64
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Comment 3 Radek Bíba 2022-10-29 07:34:58 UTC
Please try again.

Comment 4 Todd Pfrommer 2022-10-29 18:13:22 UTC
This is a Red Hat content issue. Red Hat RHUI ops has implemented a workaround in production and confirmed it resolves our reproducer. Please test the fix on the AWS side, @peninm

Comment 5 Srinivastp 2022-10-29 19:05:53 UTC
This issue is present in eu-central-1 region too.

[root@ec2-user]# yum check-update|grep NetworkManager
NetworkManager.aarch64             1:1.36.0-9.el8_6                            rhel-8-baseos-rhui-rpms
NetworkManager-cloud-setup.aarch64 1:1.36.0-7.el8_6                            rhel-8-appstream-rhui-rpms
NetworkManager-libnm.aarch64       1:1.36.0-9.el8_6                            rhel-8-baseos-rhui-rpms
NetworkManager-team.aarch64        1:1.36.0-9.el8_6                            rhel-8-baseos-rhui-rpms
NetworkManager-tui.aarch64         1:1.36.0-9.el8_6                            rhel-8-baseos-rhui-rpms
[root@ec2-user]# date
Sat Oct 29 18:56:32 UTC 2022
[root@ec2-user]#

[root@ec2-user]# curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document|grep region
  "region" : "eu-central-1",
[root@ec2-user]#
[root@ec2-user]# yum update
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered with an entitlement server. You can use subscription-manager to register.

Last metadata expiration check: 0:13:30 ago on Sat 29 Oct 2022 06:50:52 PM UTC.
Error:
 Problem 1: package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64 requires NetworkManager = 1:1.36.0-7.el8_6, but none of the providers can be installed
  - cannot install both NetworkManager-1:1.36.0-9.el8_6.aarch64 and NetworkManager-1:1.36.0-7.el8_6.aarch64
  - cannot install the best update candidate for package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64
  - cannot install the best update candidate for package NetworkManager-1:1.36.0-4.el8.aarch64
 Problem 2: problem with installed package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64
  - package NetworkManager-cloud-setup-1:1.36.0-4.el8.aarch64 requires NetworkManager-libnm(aarch-64) = 1:1.36.0-4.el8, but none of the providers can be installed
  - package NetworkManager-cloud-setup-1:1.36.0-7.el8_6.aarch64 requires NetworkManager-libnm(aarch-64) = 1:1.36.0-7.el8_6, but none of the providers can be installed
  - cannot install both NetworkManager-libnm-1:1.36.0-9.el8_6.aarch64 and NetworkManager-libnm-1:1.36.0-4.el8.aarch64
  - cannot install both NetworkManager-libnm-1:1.36.0-9.el8_6.aarch64 and NetworkManager-libnm-1:1.36.0-7.el8_6.aarch64
  - cannot install the best update candidate for package NetworkManager-libnm-1:1.36.0-4.el8.aarch64
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
[root@ec2-user]#

Comment 6 peninm 2022-10-30 02:29:53 UTC
Just tried it and still getting the same issue in us-west-2

Comment 7 Radek Bíba 2022-10-30 07:41:41 UTC
Right. The CDN repodata issue with RHEL 8 for ARM64 AppStream has been worked around in us-east-1, so VMs in the (AFAICT) most exposed region can continue to receive updates. Rather than working around the issue in all regions on the RHUI level, we aim to resolve the upstream issue ASAP.

Comment 8 Radek Bíba 2022-10-31 16:00:13 UTC
OK, the repo is now up to date in all regions. Please try one more time.

Comment 9 peninm 2022-10-31 16:24:37 UTC
Just tested it in several different regions and it was working. I'll keep checking the other regions, but looks like the update worked.


Note You need to log in before you can comment on or make changes to this bug.