Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2002077

Summary: Leapp upgrade from rhel-7.9 to 8.2 using satellite failed
Product: Red Hat Enterprise Linux 7 Reporter: shaju <shajuvk>
Component: leappAssignee: Leapp team <leapp-notifications>
Status: CLOSED NOTABUG QA Contact: Alois Mahdal <amahdal>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 7.9CC: bsanford, ggalkin, jpretori, mburns, pstodulk
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-09-08 11:31:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
leapp-upgrade-log
none
leapp-preupgrade.log none

Description shaju 2021-09-07 20:37:32 UTC
Created attachment 1821341 [details]
leapp-upgrade-log

Created attachment 1821341 [details]
leapp-upgrade-log

Description of problem:
As part of FFU process. While upgrading the undercloud from OSP13 + rhel-7.9 to rhel-8.2 , the leap upgrade command failed.

Command used.
====
sudo leapp upgrade --debug \
  --enablerepo rhel-8-for-x86_64-baseos-rpms \
  --enablerepo rhel-8-for-x86_64-appstream-rpms \
  --enablerepo rhel-8-for-x86_64-highavailability-rpms \
  --enablerepo fast-datapath-for-rhel-8-x86_64-rpms \
  --enablerepo ansible-2.9-for-rhel-8-x86_64-rpms \
  --enablerepo openstack-16.1-for-rhel-8-x86_64-rpms \
  --enablerepo satellite-tools-6.5-for-rhel-8-x86_64-rpms \
  --enablerepo advanced-virt-for-rhel-8-x86_64-rpms

Version-Release number of selected component (if applicable):

===

_/work']
2021-09-07 16:07:04.631 DEBUG    PID: 32657 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has started: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/system_overlay']
2021-09-07 16:07:04.848 DEBUG    PID: 32657 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has finished: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/system_overlay']
2021-09-07 16:07:04.906 DEBUG    PID: 32657 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Cleaning up mounts
2021-09-07 16:07:04.965 DEBUG    PID: 32657 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Recursively removing scratch directory /var/lib/leapp/scratch.
2021-09-07 16:07:05.48  DEBUG    PID: 32657 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Recursively removed scratch directory /var/lib/leapp/scratch.
Loaded plugins: product-id, search-disabled-repos, subscription-manager
Cleaning repos: advanced-virt-for-rhel-8-x86_64-rpms
              : ansible-2.9-for-rhel-8-x86_64-rpms
              : fast-datapath-for-rhel-8-x86_64-rpms
              : openstack-16.1-for-rhel-8-x86_64-rpms
              : rhel-8-for-x86_64-appstream-eus-rpms
              : rhel-8-for-x86_64-baseos-eus-rpms
              : rhel-8-for-x86_64-highavailability-eus-rpms
              : satellite-tools-6.5-for-rhel-8-x86_64-rpms
Other repos take up 98 M of disk space (use --verbose for details)
/etc/yum.repos.d/redhat.repo
/etc/yum.repos.d/redhat.repo
2021-09-07 16:07:05.265 INFO     PID: 26248 leapp.workflow.TargetTransactionFactsCollection: Starting stage After of phase TargetTransactionFactsCollection
2021-09-07 16:07:05.481 INFO     PID: 26248 leapp.workflow: Workflow interrupted due to the FailPhase error policy
2021-09-07 16:07:05.549 INFO     PID: 26248 leapp: Answerfile will be created at /var/log/leapp/answerfile
2021-09-07 16:07:05.614 WARNING  PID: 26248 leapp.reporting: Stable Key report entry not provided, dynamically generating one - 52065c151e75a7d4ef642d05df51efec432d0226
2021-09-07 16:07:05.768 WARNING  PID: 26248 leapp.reporting: Stable Key report entry not provided, dynamically generating one - 52065c151e75a7d4ef642d05df51efec432d0226

============================================================
                           ERRORS
============================================================

2021-09-07 16:07:05.107079 [ERROR] Actor: target_userspace_creator
Message: Some required custom target repositories are not available.
Summary:
    Hint:  The most probably you are using custom or third party actor that produces CustomTargetRepository message or you did a typo in one of repoids specified on command line for the leapp --enablerepo option. Inside the upgrade container, we are not able to find such repository inside any repository file. Consider use of the custom repository file regarding the official upgrade documentation or check whether you did not do a typo in any repoids you specified for the --enablerepo option of leapp.

============================================================
                       END OF ERRORS
============================================================


Debug output written to /var/log/leapp/leapp-upgrade.log

============================================================
                           REPORT
============================================================

A report has been generated at /var/log/leapp/leapp-report.json
A report has been generated at /var/log/leapp/leapp-report.txt

============================================================
                       END OF REPORT
============================================================

Answerfile has been generated at /var/log/leapp/answerfile

How reproducible:

undercloud running with 7.9 and upgrade using satellite to rhel-8 using leapp command


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

All checks from https://access.redhat.com/solutions/5233701 were done, but the issue persists.

Comment 3 Petr Stodulka 2021-09-08 11:31:35 UTC
Hi. It's clear, from the CLI there are required repositories that are not provided via satellite. E.g.
    rhel-8-for-x86_64-baseos-rpms != rhel-8-for-x86_64-baseos-eus-rpms

Repositories required via the --enablerepo option has to be available for the upgrade otherwise it's stopped.

Comment 4 Jesse Pretorius 2021-09-08 11:37:12 UTC
Thank you for noticing the subtle difference.

In https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/framework_for_upgrades_13_to_16.1/upgrading-the-undercloud-operating-system#performing-a-leapp-upgrade-on-the-undercloud we specify:

$ sudo -E leapp upgrade --debug --enablerepo rhel-8-for-x86_64-baseos-eus-rpms --enablerepo rhel-8-for-x86_64-appstream-eus-rpms ...

The instructions were not correctly followed, which confirms the diagnosis.

Comment 5 Gleb Galkin 2021-09-08 12:40:05 UTC
Created attachment 1821519 [details]
leapp-preupgrade.log

Comment 6 Gleb Galkin 2021-09-08 12:41:30 UTC
The command with correct eus repos still fails with the same error

sudo leapp preupgrade --debug --enablerepo rhel-8-for-x86_64-baseos-eus-rpms --enablerepo rhel-8-for-x86_64-appstream-eus-rpms --enablerepo rhel-8-for-x86_64-highavailability-eus-rpms --enablerepo fast-datapath-for-rhel-8-x86_64-rpms --enablerepo ansible-2.9-for-rhel-8-x86_64-rpm


Please see at attached leapp-preupgrade.log

Comment 7 Petr Stodulka 2021-09-08 12:56:16 UTC
Hi. I am pretty sure you have still at least one repository wrong. Attach please the /var/lib/leapp/leapp.db file.

Comment 8 Petr Stodulka 2021-09-08 13:06:12 UTC
But if it is not copy paste error in the shared command, I would say you are missing "s" in the last repository:
    ansible-2.9-for-rhel-8-x86_64-rpm != ansible-2.9-for-rhel-8-x86_64-rpms

We will add the list of the missing repositories into the error message to make searching of typos easier.

Comment 9 Gleb Galkin 2021-09-08 14:21:25 UTC
OK  it was actiually a typo. Missing "s" in the last repository cause the error.

Comment 10 Gleb Galkin 2021-09-08 15:12:36 UTC
I think that leapp should be more precise with the error message. If leapp would say that repoid 'ansible-2.9-for-rhel-8-x86_64-rpm' is wrong and can't be reached it will save time for me and for RedHat support.

Comment 11 Bill Sanford 2021-10-19 17:01:05 UTC
I have just tested a fix where if there is a missing or misspelled repo name there will be an inhibitor thrown and the error message will be ""Some required custom target repositories have not been found."