Bug 2143451

Summary: Satellite upgrades should not require enabling the next versions Satellite repository, and should rely only on the Maintenance repository
Product: Red Hat Satellite Reporter: Matthew Yee <myee>
Component: PackagingAssignee: Eric Helms <ehelms>
Status: CLOSED ERRATA QA Contact: Lukas Pramuk <lpramuk>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.12.0CC: ahumbe, bhoppus, brant.evans, dalley, dhjoshi, egolov, ehelms, gkeegan, hakon.gislason, ikke, j.becker, jbhatia, jyoung, lpramuk, mkalyat, momran, pcfe, pcreech, pdwyer, saydas, vcojot
Target Milestone: 6.13.0Keywords: PrioBumpGSS, Triaged, Upgrades
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
. Upgrade to version 6.12 fails with "Nothing to update, can't find new version of satellite-maintain." Due to a modularity issue, the `dnf` tool currently cannot detect the relevant version of the `rubygem-foreman_maintain` package. Consequently, the `satellite-maintain upgrade list-versions` command does not list Red Hat Satellite version 6.12 for running an upgrade after enabling the `satellite-maintenance-6.12-for-rhel-8-x86_64-rpms` repository, therefore Satellite Servers and Capsule Servers cannot be upgraded to the version 6.12. To work around this problem, use the below commands to enable the satellite 6.12 repositories after enabling the `satellite-maintenance-6.12-for-rhel-8-x86_64-rpms repository`. For Satellite Server: `# subscription-manager repos --enable satellite-6.12-for-rhel-8-x86_64-rpms` For Capsule Server: `# subscription-manager repos --enable satellite-capsule-6.12-for-rhel-8-x86_64-rpms` For more information, see the Knowledgebase Solution https://access.redhat.com/solutions/6986053.
Story Points: ---
Clone Of:
: 2184123 (view as bug list) Environment:
Last Closed: 2023-05-03 13:22:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Matthew Yee 2022-11-17 00:17:18 UTC
Description of problem:
satellite-maintain returns "Nothing to update, can't find new version of satellite-maintain." when attempting dry run of upgrade from 6.11 to 6.12.

Version-Release number of selected component (if applicable):
6.12

How reproducible:
Always

Steps to Reproduce:
1. Enable satellite maintenance 6.12 repo:
subscription-manager repos --enable satellite-maintenance-6.12-for-rhel-8-x86_64-rpms
2. Run satellite-maintain:
satellite-maintain upgrade check --target-version 6.12

Actual results:
[root@ip-172-31-29-4 ~]# subscription-manager repos --list-enabled
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   satellite-6.11-for-rhel-8-x86_64-rpms
Repo Name: Red Hat Satellite 6.11 for RHEL 8 x86_64 (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/satellite/6.11/os
Enabled:   1

Repo ID:   rhel-8-for-x86_64-appstream-rpms
Repo Name: Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/os
Enabled:   1

Repo ID:   rhel-8-for-x86_64-baseos-rpms
Repo Name: Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/os
Enabled:   1

Repo ID:   satellite-maintenance-6.12-for-rhel-8-x86_64-rpms
Repo Name: Red Hat Satellite Maintenance 6.12 for RHEL 8 x86_64 (RPMS)
Repo URL:  https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/sat-maintenance/6.12/os
Enabled:   1

Repo ID:   satellite-maintenance-6.11-for-rhel-8-x86_64-rpms
Repo Name: Red Hat Satellite Maintenance 6.11 for RHEL 8 x86_64 (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/sat-maintenance/6.11/os
Enabled:   1

[root@ip-172-31-29-4 ~]# satellite-maintain upgrade check --target-version 6.12
Checking for new version of satellite-maintain...
Security: kernel-core-4.18.0-425.3.1.el8.x86_64 is an installed security update
Security: kernel-core-4.18.0-372.32.1.el8_6.x86_64 is the currently running version
Nothing to update, can't find new version of satellite-maintain.
Can't upgrade to 6.12
Possible target versions are:
6.11.z

Expected results:
6.12 is available.

Comment 1 Eric Helms 2022-11-17 01:14:51 UTC
The workaround is to enable the satellite 6.12 repository (or Capsule repository if it's a Capsule) after enabling the maintenance repository:

subscription-manager repos --enable satellite-maintenance-6.12-for-rhel-8-x86_64-rpms
subscription-manager repos --enable satellite-6.12-for-rhel-8-x86_64-rpms

Then upgrade steps should be followed from Step 6 onward (https://access.redhat.com/documentation/en-us/red_hat_satellite/6.12/html/upgrading_and_updating_red_hat_satellite/upgrading_satellite).



The problem comes from missing module metadata in the Satellite Maintenance repository that prevents dnf from seeing that there is a newer version of rubygem-foreman_maintain available.

Comment 2 Evgeni Golov 2022-11-17 09:17:03 UTC
(In reply to Eric Helms from comment #1)

> The problem comes from missing module metadata in the Satellite Maintenance
> repository that prevents dnf from seeing that there is a newer version of
> rubygem-foreman_maintain available.

I don't this is entirely correct (but is still ultimately the cause for the issue at hand).

The problem is that rubygem-foreman_maintain (and other packages from the maintenance repo) are *ALSO* present in the Satellite and Capsule repositories.
Due to this fact, DNF considers these packages as part of a module (as Satellite and Capsule *DO* ship modular metadata, listing these packages) and modular packages cannot be updated by a non-modular package (as presented in the Maintenance repository).

This wasn't a problem for 6.11, as there the version in Satellite/Capsule and Maintenance was identical, but is now as we're telling people to only enable 6.12 Maintenance and let the tool do the rest for them.

Another (no idea if better) workaround would be to set the module_hotfixes flag on the Maintenance repository:
# subscription-manager repo-override --repo=satellite-maintenance-6.12-for-rhel-8-x86_64-rpms --add=module_hotfixes:1

We can add modular metadata to the Maintenance repository, but that will mean users will have to explicitly enable that module too, which feels kinda wrong (another step to do manually) and will also require adjustments to the Maintenance code (for the self-upgrade feature).

IMHO the most correct way would be to explicitly mark the Maintenance related packages as "demodularized" in the Satellite/Capsule modules (https://github.com/fedora-modularity/libmodulemd/blob/2edaf37c260804824678c0c5ad9e592a19bafca6/yaml_specs/modulemd_stream_v2.yaml#L387-L405) and exclude them from being included in the "artifacts.rpms" list.

Comment 4 Eric Helms 2022-11-23 13:48:56 UTC
(In reply to Eric Helms from comment #1)
> The workaround is to enable the satellite 6.12 repository (or Capsule
> repository if it's a Capsule) after enabling the maintenance repository:
> 
> subscription-manager repos --enable
> satellite-maintenance-6.12-for-rhel-8-x86_64-rpms
> subscription-manager repos --enable satellite-6.12-for-rhel-8-x86_64-rpms
> 
> Then upgrade steps should be followed from Step 6 onward
> (https://access.redhat.com/documentation/en-us/red_hat_satellite/6.12/html/
> upgrading_and_updating_red_hat_satellite/upgrading_satellite).
> 
> 
> 
> The problem comes from missing module metadata in the Satellite Maintenance
> repository that prevents dnf from seeing that there is a newer version of
> rubygem-foreman_maintain available.

One thing I would like to note to be aware of with this work around -- this enables the 6.12 server repository (or Capsule if upgrading a Capsule) and thus makes available package updates. If you enable the repositories, and run into issues as part of 'upgrade 
check' you must ensure you do not yourself or have tooling that runs yum update or that you disable the 'satellite-6.12-for-rhel-8-x86_64-rpms' repository while the errors from upgrade check are being investigated.

Comment 8 Lukas Pramuk 2022-11-25 13:03:51 UTC
FailedQA.

@Satellite 6.11.4.2 Snap1

No satellite module at internal repo /pulp/repos/Sat6-CI/6_11_4_2-1_0/Satellite_6_11_with_RHEL8_Server/custom/Satellite_6_11_Composes/Satellite_6_11_RHEL8/ !!!
Without modularity information in the satellite repo I am not able to test de-modularization of maintenance packages

>>> everything is demodularize - snap templatization is failing to produce 6.11.4.2 el8 template

Comment 14 Eric Helms 2022-12-12 21:24:36 UTC
*** Bug 2144057 has been marked as a duplicate of this bug. ***

Comment 20 Lukas Pramuk 2023-03-21 11:45:42 UTC
VERIFIED.

@Satellite 6.13.0 Snap15
rubygem-foreman_maintain-1.2.7-1.el8sat.noarch

by the following reproducers:


#1 INSTALLATION

1) Subscribe and enable required repositories (+ internal candidate repos)
2) Enable satellite:el8 module
# dnf module enable satellite:el8
...
Enabling module streams:
 pki-core                                                                   10.6                                                                                                                                           
 pki-deps                                                                   10.6                                                                                                                                           
 postgresql                                                                 12                                                                                                                                             
 ruby                                                                       2.7                                                                                                                                            
 satellite                                                                  el8                                                                                                                                            
 satellite-maintenance                                                      el8                                                                                                                                            

>>> satellite-maintenance module gets enabled automatically as a dependency of satellite module

3) Install satellite rpms
# dnf install satellite 
4) Run installer
# satellite-installer -S satellite


#2 UPGRADE

1) Have a Satellite 6.12 subscribed with required repos enabled

2) Enable Satellite 6.13 Maintenance repo (internal candidate repo atm)

3) Check whether both product modules are available 
# dnf module list satellite satellite-maintenance
...
Satellite Maintenance 6.13 RHEL8
Name                            Stream           Profiles          Summary                              
satellite-maintenance           el8                                Satellite Maintenance module         

Red Hat Satellite 6.12 for RHEL 8 x86_64 (RPMs)
Name                            Stream           Profiles          Summary                              
satellite                       el8 [e]                            Satellite module                     

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

>>> there is now new satellite-maintenance module provided by 6.13 maintenance repository

4) Enable satellite-maintenance module in order to get updates
Note: this is required only once - during next upgrade this step won't needed (vs. enabling next satellite repo as the workaround was required during every upgrade)
# dnf module enable satellite-maintenance:el8

5) Try to update maintenance rubygem 
# satellite-maintain upgrade list-versions
Checking for new version of satellite-maintain...

rubygem-foreman_maintain.noarch 1:1.2.7-1.el8sat Satellite_Engineering_Satellite_6_13_Composes_Satellite_Maintenance_6_13_RHEL8

Updating satellite-maintain package.

# satellite-maintain upgrade list-versions
Checking for new version of satellite-maintain...
Nothing to update, can't find new version of satellite-maintain.
6.12.z
6.13

>>> rubygem-foreman_maintain is successfully updated from next version repo

Comment 23 errata-xmlrpc 2023-05-03 13:22:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097

Comment 24 Red Hat Bugzilla 2023-09-19 04:30:09 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days