Bug 1569491

Summary: rhn-migrate-classic-to-rhsm should try to resolve product ID collisions detected from subscription-manager-migration-data
Product: Red Hat Enterprise Linux 7 Reporter: Kenny Tordeurs <ktordeur>
Component: subscription-managerAssignee: candlepin-bugs
Status: CLOSED ERRATA QA Contact: Red Hat subscription-manager QE Team <rhsm-qe>
Severity: high Docs Contact:
Priority: high    
Version: 7.5CC: akarimi, anerurka, candlepin-bugs, csnyder, este, gary.luciano, jalviso, janarula, jhnidek, jsefler, kagarwal, khowell, libbe, mkalyat, rajukuma, rbeyel, redakkan, rjerrido, roarora, smajumda, sraut, susalvi, wpoteat
Target Milestone: rcKeywords: SubscriptionExperience, Triaged
Target Release: 7.9Flags: rjerrido: needinfo-
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: subscription-manager-1.24.29-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-29 19:22:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1549766, 1822749    
Bug Blocks: 1796188    

Description Kenny Tordeurs 2018-04-19 11:51:25 UTC
Description of problem:

Migration with bootstrap.py for Satellite 5 clients on RHEL7 Server or Workstation to Satellite 6 is failing with following errors:

~~~
2018-04-16 14:17:58,120 [INFO] rhn-migrate-classic-to-rhsm:10207:MainThread @migrate.py:474 - Using mapping file /usr/share/rhsm/product/RHEL-7/channel-cert-mapping.txt
2018-04-16 14:17:58,143 [INFO] rhn-migrate-classic-to-rhsm:10207:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-workstation-7 = Workstation-Workstation-x86_64-e2e5f826adb1-71.pem
2018-04-16 14:17:58,144 [INFO] rhn-migrate-classic-to-rhsm:10207:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-workstation-extras-7 = Workstation-Workstation-x86_64-30b4e57a8843-71.pem
2018-04-16 14:17:58,144 [INFO] rhn-migrate-classic-to-rhsm:10207:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-workstation-supplementary-7 = Workstation-Workstation-x86_64-06b84075cc00-71.pem
2018-04-16 14:17:58,144 [INFO] rhn-migrate-classic-to-rhsm:10207:MainThread @migrate.py:495 - Mapping found for: rhn-tools-rhel-x86_64-workstation-7 = Workstation-Workstation-x86_64-06b84075cc00-71.pem
2018-04-16 14:17:58,144 [INFO] rhn-migrate-classic-to-rhsm:10207:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-workstation-optional-7 = Workstation-Workstation-x86_64-06b84075cc00-71.pem
2018-04-16 14:17:58,145 [ERROR] rhn-migrate-classic-to-rhsm:10207:MainThread @migrate.py:458 - Aborting. Detected the following product ID collisions: {'71': {'Workstation-Workstation-x86_64-e2e5f826adb1-71.pem': ['rhel-x86_64-workstation-7'], 'Workstation-Workstation-x86_64-30b4e57a8843-71.pem': ['rhel-x86_64-workstation-extras-7'], 'Workstation-Workstation-x86_64-06b84075cc00-71.pem': ['rhel-x86_64-workstation-supplementary-7', 'rhn-tools-rhel-x86_64-workstation-7', 'rhel-x86_64-workstation-optional-7']}}
~~~

~~~
2018-04-12 19:02:47,309 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:474 - Using mapping file /usr/share/rhsm/product/RHEL-7/channel-cert-mapping.txt
2018-04-12 19:02:47,318 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-server-7 = Server-Server-x86_64-c1aff91afa01-69.pem
2018-04-12 19:02:47,318 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-server-optional-7 = Server-Server-x86_64-4ef1264b94f3-69.pem
2018-04-12 19:02:47,318 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:495 - Mapping found for: rhn-tools-rhel-x86_64-server-7 = Server-Server-x86_64-4ef1264b94f3-69.pem
2018-04-12 19:02:47,318 [ERROR] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:458 - Aborting. Detected the following product ID collisions: {'69': {'Server-Server-x86_64-4ef1264b94f3-69.pem': ['rhel-x86_64-server-optional-7', 'rhn-tools-rhel-x86_64-server-7'], 'Server-Server-x86_64-c1aff91afa01-69.pem': ['rhel-x86_64-server-7']}}
~~~



Version-Release number of selected component (if applicable):
# grep migration installed-rpms                          
~~~
subscription-manager-migration-1.20.11-1.el7_5.x86_64       Thu Apr 12 19:02:33 2018
subscription-manager-migration-data-2.0.39-1.el7.noarch     Thu Apr 12 19:02:33 2018
~~~

How reproducible:
100%

Steps to Reproduce:
1. Install client on Satellite 5 with several RHEL7 channels like extras,supplementary,optional
2. Try to migrate the client via bootstrap.py it will install latest subscription-manager-migration* from RHEL 7.5
3. Fails on mapping

Actual results:
2018-04-12 19:02:47,309 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:474 - Using mapping file /usr/share/rhsm/product/RHEL-7/channel-cert-mapping.txt
2018-04-12 19:02:47,318 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-server-7 = Server-Server-x86_64-c1aff91afa01-69.pem
2018-04-12 19:02:47,318 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:495 - Mapping found for: rhel-x86_64-server-optional-7 = Server-Server-x86_64-4ef1264b94f3-69.pem
2018-04-12 19:02:47,318 [INFO] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:495 - Mapping found for: rhn-tools-rhel-x86_64-server-7 = Server-Server-x86_64-4ef1264b94f3-69.pem
2018-04-12 19:02:47,318 [ERROR] rhn-migrate-classic-to-rhsm:19493:MainThread @migrate.py:458 - Aborting. Detected the following product ID collisions: {'69': {'Server-Server-x86_64-4ef1264b94f3-69.pem': ['rhel-x86_64-server-optional-7', 'rhn-tools-rhel-x86_64-server-7'], 'Server-Server-x86_64-c1aff91afa01-69.pem': ['rhel-x86_64-server-7']}}


Expected results:
rhsm.log
~~~
2018-04-16 17:42:58,746 [INFO] rhn-migrate-classic-to-rhsm:2111:MainThread @migrate.py:472 - Using mapping file /usr/share/rhsm/product/RHEL-7/channel-cert-mapping.txt
2018-04-16 17:42:58,752 [INFO] rhn-migrate-classic-to-rhsm:2111:MainThread @migrate.py:493 - Mapping found for: rhel-x86_64-server-7 = Server-Server-x86_64-89d5cb5bcfe3-69.pem
2018-04-16 17:42:58,753 [INFO] rhn-migrate-classic-to-rhsm:2111:MainThread @migrate.py:493 - Mapping found for: rhn-tools-rhel-x86_64-server-7 = Server-Server-x86_64-89d5cb5bcfe3-69.pem
2018-04-16 17:42:58,753 [INFO] rhn-migrate-classic-to-rhsm:2111:MainThread @migrate.py:493 - Mapping found for: rhel-x86_64-server-optional-7 = Server-Server-x86_64-89d5cb5bcfe3-69.pem
2018-04-16 17:42:58,753 [INFO] rhn-migrate-classic-to-rhsm:2111:MainThread @migrate.py:526 - Certs to be installed: {'69': {'Server-Server-x86_64-89d5cb5bcfe3-69.pem': ['rhel-x86_64-server-7', 'rhn-tools-rhel-x86_64-server-7', 'rhel-x86_64-server-optional-7']}}
~~~

[root@ktordeur-rhel7-client-sat57 ~]#  egrep "^rhel-x86_64-server-7:|^rhel-x86_64-server-optional-7:|^rhn-tools-rhel-x86_64-server-7:" /usr/share/rhsm/product/RHEL-7/channel-cert-mapping.txt
~~~
rhel-x86_64-server-7: Server-Server-x86_64-89d5cb5bcfe3-69.pem
rhel-x86_64-server-optional-7: Server-Server-x86_64-89d5cb5bcfe3-69.pem
rhn-tools-rhel-x86_64-server-7: Server-Server-x86_64-89d5cb5bcfe3-69.pem
~~~

This works with older version from RHEL 7.4
# rpm -qa | grep subscription-manager
~~~
subscription-manager-migration-1.19.23-1.el7_4.x86_64
subscription-manager-migration-data-2.0.37-1.el7.noarch
subscription-manager-1.19.23-1.el7_4.x86_64
~~~

Additional info:

Can be seen when I install the packages from RHEL7.5 fails with same error:

# rpm -qa | grep subscription-manager-mig
~~~
subscription-manager-migration-data-2.0.39-1.el7.noarch
subscription-manager-migration-1.20.11-1.el7_5.x86_64
~~~

[root@ktordeur-rhel7-client-sat57 ~]# python bootstrap.py -l admin -s provisioning.sysmgmt.lan -o 'Default Organization' -L 'Default Location' -g RHEL7 -a RHEL7_Library --force
Foreman Bootstrap Script
This script is designed to register new systems or to migrate an existing system to a Foreman server with Katello
admin's password:
[NOTIFICATION], [2018-04-19 15:10:04], [This system is registered to RHN. Attempting to migrate via rhn-classic-migrate-to-rhsm] 
[NOTIFICATION], [2018-04-19 15:10:04], [Installing subscription manager prerequisites] 
[RUNNING], [2018-04-19 15:10:04], [/usr/bin/yum -y remove subscription-manager-gnome] 
Loaded plugins: enabled_repos_upload, package_upload, product-id, rhnplugin,
              : search-disabled-repos, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
No Match for argument: subscription-manager-gnome
No Packages marked for removal
Uploading Enabled Repositories Report
Loaded plugins: product-id, rhnplugin, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
Cannot upload enabled repos report, is this client registered?
[SUCCESS], [2018-04-19 15:10:06], [/usr/bin/yum -y remove subscription-manager-gnome], completed successfully.

[RUNNING], [2018-04-19 15:10:06], [/usr/bin/yum -y install subscription-manager 'subscription-manager-migration-*'] 
Loaded plugins: enabled_repos_upload, package_upload, product-id, rhnplugin,
              : search-disabled-repos, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
Package matching subscription-manager-1.19.23-1.el7_4.x86_64 already installed. Checking for update.
Package matching subscription-manager-migration-1.19.23-1.el7_4.x86_64 already installed. Checking for update.
Package matching subscription-manager-migration-data-2.0.37-1.el7.noarch already installed. Checking for update.
Nothing to do
Uploading Enabled Repositories Report
Loaded plugins: product-id, rhnplugin, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
Cannot upload enabled repos report, is this client registered?
[SUCCESS], [2018-04-19 15:10:09], [/usr/bin/yum -y install subscription-manager 'subscription-manager-migration-*'], completed successfully.

[RUNNING], [2018-04-19 15:10:09], [/usr/bin/yum -y update yum openssl python] 
Loaded plugins: enabled_repos_upload, package_upload, product-id, rhnplugin,
              : search-disabled-repos, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
No packages marked for update
Uploading Enabled Repositories Report
Loaded plugins: product-id, rhnplugin, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
Cannot upload enabled repos report, is this client registered?
[SUCCESS], [2018-04-19 15:10:10], [/usr/bin/yum -y update yum openssl python], completed successfully.

[NOTIFICATION], [2018-04-19 15:10:10], [Removing old Katello agent and certs] 
[SUCCESS], [2018-04-19 15:10:10], [Removing /etc/rhsm/ca/katello-server-ca.pem], completed successfully.
[RUNNING], [2018-04-19 15:10:10], [/usr/bin/yum -y erase 'katello-ca-consumer-*' katello-agent gofer] 
Loaded plugins: enabled_repos_upload, package_upload, product-id, rhnplugin,
              : search-disabled-repos, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
No Match for argument: katello-agent
No Match for argument: gofer
Resolving Dependencies
--> Running transaction check
---> Package katello-ca-consumer-provisioning.sysmgmt.lan.noarch 0:1.0-5 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                                       Arch    Version Repository  Size
================================================================================
Removing:
 katello-ca-consumer-provisioning.sysmgmt.lan  noarch  1.0-5   installed   14 k

Transaction Summary
================================================================================
Remove  1 Package

Installed size: 14 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Erasing    : katello-ca-consumer-provisioning.sysmgmt.lan-1.0-5.noarch    1/1 
Uploading Package Profile
  Verifying  : katello-ca-consumer-provisioning.sysmgmt.lan-1.0-5.noarch    1/1 

Removed:
  katello-ca-consumer-provisioning.sysmgmt.lan.noarch 0:1.0-5                   

Complete!
Uploading Enabled Repositories Report
Loaded plugins: product-id, rhnplugin, subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
Cannot upload enabled repos report, is this client registered?
[SUCCESS], [2018-04-19 15:10:19], [/usr/bin/yum -y erase 'katello-ca-consumer-*' katello-agent gofer], completed successfully.

[NOTIFICATION], [2018-04-19 15:10:19], [Retrieving Client CA Certificate RPMs] 
[RUNNING], [2018-04-19 15:10:19], [rpm -Uvh http://provisioning.sysmgmt.lan/pub/katello-ca-consumer-latest.noarch.rpm] 
Retrieving http://provisioning.sysmgmt.lan/pub/katello-ca-consumer-latest.noarch.rpm
Preparing...                          ########################################
Updating / installing...
katello-ca-consumer-provisioning.sysmg########################################
[SUCCESS], [2018-04-19 15:10:22], [rpm -Uvh http://provisioning.sysmgmt.lan/pub/katello-ca-consumer-latest.noarch.rpm], completed successfully.

[RUNNING], [2018-04-19 15:10:28], [Calling Foreman API to create a host entry associated with the group & org] 
[SUCCESS], [2018-04-19 15:10:33], [Successfully created host ktordeur-rhel7-client-sat57], completed successfully.
[NOTIFICATION], [2018-04-19 15:10:33], [Calling rhn-migrate-classic-to-rhsm] 
[RUNNING], [2018-04-19 15:10:33], [/usr/sbin/rhn-migrate-classic-to-rhsm --org Default_Organization --activation-key 'RHEL7_Library'  --force --destination-url=https://provisioning.sysmgmt.lan:443/rhsm --keep] 
[ERROR], [2018-04-19 15:10:34], EXITING: [/usr/sbin/rhn-migrate-classic-to-rhsm --org Default_Organization --activation-key 'RHEL7_Library'  --force --destination-url=https://provisioning.sysmgmt.lan:443/rhsm --keep] failed to execute properly.

Retrieving existing legacy subscription information...

+-----------------------------------------------------+
System is currently subscribed to these legacy channels:
+-----------------------------------------------------+
rhel-x86_64-server-7
rhel-x86_64-server-7-rhscl-1
rhel-x86_64-server-optional-7
rhn-tools-rhel-x86_64-server-7

+-----------------------------------------------------+
Unable to continue migration!
+-----------------------------------------------------+
You are subscribed to channels that have conflicting product certificates.
The following channels map to product ID 69:
	rhel-x86_64-server-7
	rhel-x86_64-server-optional-7
	rhn-tools-rhel-x86_64-server-7
Reduce the number of channels per product ID to 1 and run migration again.
To remove a channel, use 'rhn-channel --remove --channel=<conflicting_channel>'.

Comment 2 Kenny Tordeurs 2018-04-19 12:13:34 UTC
The following workaround should work:

Downgrade the subscription-manager* packages to previous version:
~~~
# yum downgrade subscription-manager* -y
~~~

~~~
# rpm -qa | grep subscription-manager | sort

subscription-manager-1.20.10-1.el7.x86_64
subscription-manager-migration-1.20.10-1.el7.x86_64
subscription-manager-migration-data-2.0.37-1.el7.noarch
~~~

Comment 3 Kevin Howell 2018-04-23 14:38:24 UTC
Server-Server-x86_64-4ef1264b94f3-69.pem is a 7.4 product certificate, while Server-Server-x86_64-c1aff91afa01-69.pem is a 7.5 product certificate. There are currently some known issues with RHN Channel mappings.

Comment 4 John Sefler 2018-04-23 15:04:02 UTC
This bug is really blocked by bug 1549766 which depends on RCMWORK-7751
Alternatively, you could follow the suggestion from last few lines of comment 0 which came from the output of rhn-migrate-classic-to-rhsm...

+-----------------------------------------------------+
Unable to continue migration!
+-----------------------------------------------------+
You are subscribed to channels that have conflicting product certificates.
The following channels map to product ID 69:
	rhel-x86_64-server-7
	rhel-x86_64-server-optional-7
	rhn-tools-rhel-x86_64-server-7
Reduce the number of channels per product ID to 1 and run migration again.
To remove a channel, use 'rhn-channel --remove --channel=<conflicting_channel>'.

I'd suggest removing channels rhel-x86_64-server-optional-7 and rhn-tools-rhel-x86_64-server-7 and then running the migration again.

Comment 7 Rich Jerrido 2018-06-13 21:27:17 UTC
Since the channels in question are provided by the baseOS' product cert, a workaround for customers who are using bootstrap.py would be to pass the '--skip migration' option, which does not run rhn-migrate-classic-to-rhsm, but merely disables the RHN plugin to yum, and performs a subscription-manager register. 

Assuming that they are on a modern-ish version of RHEL, the product cert for the baseOS is included in the redhat-release-* package, so running rhn-migrate-classic-to-rhsm is somewhat superfluous.

Comment 8 John Sefler 2018-06-15 13:51:46 UTC
Changing the accuracy of the bug title because "rhn-migrate-classic-to-rhsm fails with detected the following product ID collisions" is not right.  In fact, rhn-migrate-classic-to-rhsm successfully detected product ID collisions and then decided to abort because it did not want to guess how to resolve the conflict.  Instead it left the user with a suggestion for manual resolution as shown in comment 4.  But as Kenny points out in comment 5, this is not a solution when migrating many client using the bootstrap.py wrapper.

Here's a proposal for a more satisfactory way for rhn-migrate-classic to handle the case of product ID collisions (to benefit bootstrap.py)...

When rhn-migrate-classic detects a product ID collision, it could....
  Compare the properties of the migration product certs...
    1. Product ID
    2. Version
    3. Arch
    4. Tags
  If the only difference between the colliding migration product certs is the Version property, then disregard product cert with the older version.  Just ignore it.  Don't install it.

Since all the properties of the the newer versioned product cert will be the same, subscription-manager --auto-attach and yum access to content will be the same.  In fact, once the system starts executing yum transactions after the migration is complete, the product id cert will actually correct itself with the latest productid certs from the repodata on the CDN.

Comment 23 John Sefler 2020-04-13 23:27:29 UTC
Reproducing a classic rhn registration that will yield a "conflicting product certificates" situation on a rhel-7.8 system...

[root@kvm-06-guest02 ~]# rpm -q subscription-manager-migration
subscription-manager-migration-1.24.26-1.el7.x86_64
[root@kvm-06-guest02 ~]# rpm -q subscription-manager-migration-data
subscription-manager-migration-data-2.0.50-1.el7.noarch

Before I begin, I will show that the only product cert installed is the product-default provided by package redhat-release-server-7.8...
[root@kvm-06-guest02 ~]# ls /etc/pki/product*
/etc/pki/product:

/etc/pki/product-default:
69.pem
[root@kvm-06-guest02 ~]# rpm -q --whatprovides /etc/pki/product-default/69.pem 
redhat-release-server-7.8-2.el7.x86_64


To reproduce the offending situation in comment 0, I will use rhnreg_ks to register classically and add many many many rhn-channels.


[root@kvm-06-guest02 ~]# rhnreg_ks --serverUrl=https://rhsm-sat58.usersys.redhat.com/XMLRPC --username=REDACTED --password=REDACTED --norhnsd --nohardware --nopackages --novirtinfo
[root@kvm-06-guest02 ~]# 
[root@kvm-06-guest02 ~]# rhn-channel -u REDACTED -p REDACTED -a -c rhel-x86_64-server-7-mrg-messaging-2-debuginfo -c rhel-x86_64-server-7-mrg-messaging-2 -c rhel-x86_64-server-rh-common-7-debuginfo -c rhel-x86_64-server-rh-common-7 -c rhel-x86_64-rhev-mgmt-agent-7-debuginfo -c rhel-x86_64-rhev-mgmt-agent-7 -c rhel-x86_64-server-7-ost-5-cts -c rhel-x86_64-server-7-ost-5-debuginfo -c rhel-x86_64-server-7-ost-5 -c rhel-x86_64-server-7-ost-6-cts -c rhel-x86_64-server-7-ost-6-debuginfo -c rhel-x86_64-server-7-ost-6 -c rhel-x86_64-server-7-ost-5-cts-debuginfo -c rhel-x86_64-server-7-ost-6-cts-debuginfo -c rhel-x86_64-server-7-rhscl-1-beta-debuginfo -c rhel-x86_64-server-7-rhscl-1-beta -c rhel-x86_64-server-7-rhscl-1-debuginfo -c rhel-x86_64-server-7-rhscl-1 -c rhel-x86_64-server-rt-7-debuginfo -c rhel-x86_64-server-nfv-7-debuginfo -c rhel-x86_64-server-nfv-7 -c rhel-x86_64-server-rt-7 -c rhel-x86_64-server-sap-7-debuginfo -c rhel-x86_64-server-sap-hana-7-debuginfo -c rhel-x86_64-server-sap-hana-7 -c rhel-x86_64-server-sap-7 -c rhel-x86_64-server-7-ost-6-installer -c rhel-x86_64-server-7-ost-6-installer-debuginfo -c rhel-x86_64-server-7-ost-7-optools -c rhel-x86_64-server-7-ost-7-optools-debuginfo -c rhel-x86_64-server-7-ost-7-director -c rhel-x86_64-server-7-ost-7-director-debuginfo -c rhel-x86_64-server-7-ost-7 -c rhel-x86_64-server-7-ost-7-debuginfo -c rhel-x86_64-ost-7-tools-server-debuginfo -c rhel-x86_64-ost-7-tools-server -c rhel-x86_64-server-7-beta-debuginfo -c rhel-x86_64-server-7-beta -c rhel-x86_64-server-7-debuginfo -c rhel-x86_64-server-extras-7-debuginfo -c rhel-x86_64-server-extras-7 -c rhel-x86_64-server-ha-7-beta-debuginfo -c rhel-x86_64-server-ha-7-beta -c rhel-x86_64-server-ha-7-debuginfo -c rhel-x86_64-server-ha-7 -c rhel-x86_64-server-optional-7-beta-debuginfo -c rhel-x86_64-server-optional-7-beta -c rhel-x86_64-server-optional-7-debuginfo -c rhel-x86_64-server-optional-7 -c rhel-x86_64-server-rs-7-beta-debuginfo -c rhel-x86_64-server-rs-7-beta -c rhel-x86_64-server-rs-7-debuginfo -c rhel-x86_64-server-rs-7 -c rhel-x86_64-server-supplementary-7-beta-debuginfo -c rhel-x86_64-server-supplementary-7-beta -c rhel-x86_64-server-supplementary-7-debuginfo -c rhel-x86_64-server-supplementary-7 -c rhel-x86_64-server-v2vwin-7-debuginfo -c rhel-x86_64-server-v2vwin-7 -c rhn-tools-rhel-x86_64-server-7-debuginfo -c rhn-tools-rhel-x86_64-server-7
[root@kvm-06-guest02 ~]# 


I know from inspecting /usr/share/rhsm/product/RHEL-7/channel-cert-mapping.txt that many of these channels map to different certificate versions of the same product id file (69.pem), for example...

[root@kvm-06-guest02 ~]# egrep "^rhel-x86_64-server-7:|^rhel-x86_64-server-7-beta:" /usr/share/rhsm/product/RHEL-7/channel-cert-mapping.txt
rhel-x86_64-server-7: Server-x86_64-1c875a357a4f-69.pem
rhel-x86_64-server-7-beta: Server-x86_64-e94d3de24d6a-69.pem


Now let's attempt to run rhn-migrate-classic-to-rhsm expecting to encounter "conflicting product certificates"...


[root@kvm-06-guest02 ~]# rhn-migrate-classic-to-rhsm --destination-url=subscription.rhsm.stage.redhat.com:443/subscription
Legacy username: REDACTED
Legacy password: 
Destination username: REDACTED
Destination password: 

Retrieving existing legacy subscription information...

+-----------------------------------------------------+
System is currently subscribed to these legacy channels:
+-----------------------------------------------------+
rhel-x86_64-server-7
rhel-x86_64-server-7-mrg-messaging-2-debuginfo
rhel-x86_64-server-7-mrg-messaging-2
rhel-x86_64-server-rh-common-7-debuginfo
rhel-x86_64-server-rh-common-7
rhel-x86_64-rhev-mgmt-agent-7-debuginfo
rhel-x86_64-rhev-mgmt-agent-7
rhel-x86_64-server-7-ost-5-cts
rhel-x86_64-server-7-ost-5-debuginfo
rhel-x86_64-server-7-ost-5
rhel-x86_64-server-7-ost-6-cts
rhel-x86_64-server-7-ost-6-debuginfo
rhel-x86_64-server-7-ost-6
rhel-x86_64-server-7-ost-5-cts-debuginfo
rhel-x86_64-server-7-ost-6-cts-debuginfo
rhel-x86_64-server-7-rhscl-1-beta-debuginfo
rhel-x86_64-server-7-rhscl-1-beta
rhel-x86_64-server-7-rhscl-1-debuginfo
rhel-x86_64-server-7-rhscl-1
rhel-x86_64-server-rt-7-debuginfo
rhel-x86_64-server-nfv-7-debuginfo
rhel-x86_64-server-nfv-7
rhel-x86_64-server-rt-7
rhel-x86_64-server-sap-7-debuginfo
rhel-x86_64-server-sap-hana-7-debuginfo
rhel-x86_64-server-sap-hana-7
rhel-x86_64-server-sap-7
rhel-x86_64-server-7-ost-6-installer
rhel-x86_64-server-7-ost-6-installer-debuginfo
rhel-x86_64-server-7-ost-7-optools
rhel-x86_64-server-7-ost-7-optools-debuginfo
rhel-x86_64-server-7-ost-7-director
rhel-x86_64-server-7-ost-7-director-debuginfo
rhel-x86_64-server-7-ost-7
rhel-x86_64-server-7-ost-7-debuginfo
rhel-x86_64-ost-7-tools-server-debuginfo
rhel-x86_64-ost-7-tools-server
rhel-x86_64-server-7-beta-debuginfo
rhel-x86_64-server-7-beta
rhel-x86_64-server-7-debuginfo
rhel-x86_64-server-extras-7-debuginfo
rhel-x86_64-server-extras-7
rhel-x86_64-server-ha-7-beta-debuginfo
rhel-x86_64-server-ha-7-beta
rhel-x86_64-server-ha-7-debuginfo
rhel-x86_64-server-ha-7
rhel-x86_64-server-optional-7-beta-debuginfo
rhel-x86_64-server-optional-7-beta
rhel-x86_64-server-optional-7-debuginfo
rhel-x86_64-server-optional-7
rhel-x86_64-server-rs-7-beta-debuginfo
rhel-x86_64-server-rs-7-beta
rhel-x86_64-server-rs-7-debuginfo
rhel-x86_64-server-rs-7
rhel-x86_64-server-supplementary-7-beta-debuginfo
rhel-x86_64-server-supplementary-7-beta
rhel-x86_64-server-supplementary-7-debuginfo
rhel-x86_64-server-supplementary-7
rhel-x86_64-server-v2vwin-7-debuginfo
rhel-x86_64-server-v2vwin-7
rhn-tools-rhel-x86_64-server-7-debuginfo
rhn-tools-rhel-x86_64-server-7

+-----------------------------------------------------+
Unable to continue migration!
+-----------------------------------------------------+
You are subscribed to channels that have conflicting product certificates.
The following channels map to product ID 83:
	rhel-x86_64-server-ha-7
	rhel-x86_64-server-ha-7-beta
	rhel-x86_64-server-ha-7-beta-debuginfo
	rhel-x86_64-server-ha-7-debuginfo
The following channels map to product ID 208:
	rhel-x86_64-server-7-ost-5-cts
	rhel-x86_64-server-7-ost-5-cts-debuginfo
	rhel-x86_64-server-7-ost-6-cts
	rhel-x86_64-server-7-ost-6-cts-debuginfo
The following channels map to product ID 69:
	rhel-x86_64-ost-7-tools-server
	rhel-x86_64-ost-7-tools-server-debuginfo
	rhel-x86_64-server-7
	rhel-x86_64-server-7-beta
	rhel-x86_64-server-7-beta-debuginfo
	rhel-x86_64-server-7-debuginfo
	rhel-x86_64-server-extras-7
	rhel-x86_64-server-extras-7-debuginfo
	rhel-x86_64-server-optional-7
	rhel-x86_64-server-optional-7-beta
	rhel-x86_64-server-optional-7-beta-debuginfo
	rhel-x86_64-server-optional-7-debuginfo
	rhel-x86_64-server-rh-common-7
	rhel-x86_64-server-rh-common-7-debuginfo
	rhel-x86_64-server-supplementary-7
	rhel-x86_64-server-supplementary-7-beta
	rhel-x86_64-server-supplementary-7-beta-debuginfo
	rhel-x86_64-server-supplementary-7-debuginfo
	rhel-x86_64-server-v2vwin-7
	rhel-x86_64-server-v2vwin-7-debuginfo
	rhn-tools-rhel-x86_64-server-7
	rhn-tools-rhel-x86_64-server-7-debuginfo
The following channels map to product ID 191:
	rhel-x86_64-server-7-ost-5
	rhel-x86_64-server-7-ost-5-debuginfo
	rhel-x86_64-server-7-ost-6
	rhel-x86_64-server-7-ost-6-debuginfo
	rhel-x86_64-server-7-ost-6-installer
	rhel-x86_64-server-7-ost-6-installer-debuginfo
	rhel-x86_64-server-7-ost-7
	rhel-x86_64-server-7-ost-7-debuginfo
	rhel-x86_64-server-7-ost-7-director
	rhel-x86_64-server-7-ost-7-director-debuginfo
	rhel-x86_64-server-7-ost-7-optools
	rhel-x86_64-server-7-ost-7-optools-debuginfo
The following channels map to product ID 90:
	rhel-x86_64-server-rs-7
	rhel-x86_64-server-rs-7-beta
	rhel-x86_64-server-rs-7-beta-debuginfo
	rhel-x86_64-server-rs-7-debuginfo
Reduce the number of channels per product ID to 1 and run migration again.
To remove a channel, use 'rhn-channel --remove --channel=<conflicting_channel>'.
[root@kvm-06-guest02 ~]# 


BANG! Just reproduced the multiple "conflicting product certificates" just like the case that is blocking comment 0.


Now I will upgrade subscription-manager-migration to the latest version targeted for rhel-7.9 (not showing all the upgrade details)....


[root@kvm-06-guest02 ~]# yum update subscription-manager-migration --quiet --assumeyes
[root@kvm-06-guest02 ~]# rpm -q subscription-manager-migration
subscription-manager-migration-1.24.32-1.el7.x86_64
[root@kvm-06-guest02 ~]# 
[root@kvm-06-guest02 ~]# rpm -q subscription-manager-migration --changelog | grep 1569491 -A1
- 1569491: rhn-migrate-classic-to-rhsm should try to resolve product ID
  collisions (wpoteat)


Now that I have a fixed version of subscription-manager-migration installed, let's attempt rhn-migrate-classic-to-rhsm again (without removing the conflicting rhn-channels)....


[root@kvm-06-guest02 ~]# rhn-migrate-classic-to-rhsm --destination-url=subscription.rhsm.stage.redhat.com:443/subscription
Legacy username: REDACTED
Legacy password: 
Destination username: REDACTED
Destination password: 

Retrieving existing legacy subscription information...

+-----------------------------------------------------+
System is currently subscribed to these legacy channels:
+-----------------------------------------------------+
rhel-x86_64-server-7
rhel-x86_64-server-7-mrg-messaging-2-debuginfo
rhel-x86_64-server-7-mrg-messaging-2
rhel-x86_64-server-rh-common-7-debuginfo
rhel-x86_64-server-rh-common-7
rhel-x86_64-rhev-mgmt-agent-7-debuginfo
rhel-x86_64-rhev-mgmt-agent-7
rhel-x86_64-server-7-ost-5-cts
rhel-x86_64-server-7-ost-5-debuginfo
rhel-x86_64-server-7-ost-5
rhel-x86_64-server-7-ost-6-cts
rhel-x86_64-server-7-ost-6-debuginfo
rhel-x86_64-server-7-ost-6
rhel-x86_64-server-7-ost-5-cts-debuginfo
rhel-x86_64-server-7-ost-6-cts-debuginfo
rhel-x86_64-server-7-rhscl-1-beta-debuginfo
rhel-x86_64-server-7-rhscl-1-beta
rhel-x86_64-server-7-rhscl-1-debuginfo
rhel-x86_64-server-7-rhscl-1
rhel-x86_64-server-rt-7-debuginfo
rhel-x86_64-server-nfv-7-debuginfo
rhel-x86_64-server-nfv-7
rhel-x86_64-server-rt-7
rhel-x86_64-server-sap-7-debuginfo
rhel-x86_64-server-sap-hana-7-debuginfo
rhel-x86_64-server-sap-hana-7
rhel-x86_64-server-sap-7
rhel-x86_64-server-7-ost-6-installer
rhel-x86_64-server-7-ost-6-installer-debuginfo
rhel-x86_64-server-7-ost-7-optools
rhel-x86_64-server-7-ost-7-optools-debuginfo
rhel-x86_64-server-7-ost-7-director
rhel-x86_64-server-7-ost-7-director-debuginfo
rhel-x86_64-server-7-ost-7
rhel-x86_64-server-7-ost-7-debuginfo
rhel-x86_64-ost-7-tools-server-debuginfo
rhel-x86_64-ost-7-tools-server
rhel-x86_64-server-7-beta-debuginfo
rhel-x86_64-server-7-beta
rhel-x86_64-server-7-debuginfo
rhel-x86_64-server-extras-7-debuginfo
rhel-x86_64-server-extras-7
rhel-x86_64-server-ha-7-beta-debuginfo
rhel-x86_64-server-ha-7-beta
rhel-x86_64-server-ha-7-debuginfo
rhel-x86_64-server-ha-7
rhel-x86_64-server-optional-7-beta-debuginfo
rhel-x86_64-server-optional-7-beta
rhel-x86_64-server-optional-7-debuginfo
rhel-x86_64-server-optional-7
rhel-x86_64-server-rs-7-beta-debuginfo
rhel-x86_64-server-rs-7-beta
rhel-x86_64-server-rs-7-debuginfo
rhel-x86_64-server-rs-7
rhel-x86_64-server-supplementary-7-beta-debuginfo
rhel-x86_64-server-supplementary-7-beta
rhel-x86_64-server-supplementary-7-debuginfo
rhel-x86_64-server-supplementary-7
rhel-x86_64-server-v2vwin-7-debuginfo
rhel-x86_64-server-v2vwin-7
rhn-tools-rhel-x86_64-server-7-debuginfo
rhn-tools-rhel-x86_64-server-7
You are subscribed to channels that have conflicting product certificates.
Mapping product '83' to certificate 'Server-HighAvailability-x86_64-8ffc3983ad8f-83.pem'.
Mapping product '208' to certificate 'OpenStack-5.0-CTS-x86_64-fe7f145d82b8-208.pem'.
Mapping product '69' to certificate 'Server-Server-x86_64-02c044503896-69.pem'.
Mapping product '191' to certificate 'OpenStack-5.0-x86_64-4dbb67de7a84-191.pem'.
Mapping product '90' to certificate 'Server-ResilientStorage-x86_64-2fe91886c4e1-90.pem'.

+-----------------------------------------------------+
Installing product certificates for these legacy channels:
+-----------------------------------------------------+
rhel-x86_64-server-7
rhel-x86_64-server-7-mrg-messaging-2-debuginfo
rhel-x86_64-server-7-mrg-messaging-2
rhel-x86_64-server-rh-common-7-debuginfo
rhel-x86_64-server-rh-common-7
rhel-x86_64-rhev-mgmt-agent-7-debuginfo
rhel-x86_64-rhev-mgmt-agent-7
rhel-x86_64-server-7-ost-5-cts
rhel-x86_64-server-7-ost-5-debuginfo
rhel-x86_64-server-7-ost-5
rhel-x86_64-server-7-ost-6-cts
rhel-x86_64-server-7-ost-6-debuginfo
rhel-x86_64-server-7-ost-6
rhel-x86_64-server-7-ost-5-cts-debuginfo
rhel-x86_64-server-7-ost-6-cts-debuginfo
rhel-x86_64-server-7-rhscl-1-beta-debuginfo
rhel-x86_64-server-7-rhscl-1-beta
rhel-x86_64-server-7-rhscl-1-debuginfo
rhel-x86_64-server-7-rhscl-1
rhel-x86_64-server-rt-7-debuginfo
rhel-x86_64-server-nfv-7-debuginfo
rhel-x86_64-server-nfv-7
rhel-x86_64-server-rt-7
rhel-x86_64-server-sap-7-debuginfo
rhel-x86_64-server-sap-hana-7-debuginfo
rhel-x86_64-server-sap-hana-7
rhel-x86_64-server-sap-7
rhel-x86_64-server-7-ost-6-installer
rhel-x86_64-server-7-ost-6-installer-debuginfo
rhel-x86_64-server-7-ost-7-optools
rhel-x86_64-server-7-ost-7-optools-debuginfo
rhel-x86_64-server-7-ost-7-director
rhel-x86_64-server-7-ost-7-director-debuginfo
rhel-x86_64-server-7-ost-7
rhel-x86_64-server-7-ost-7-debuginfo
rhel-x86_64-ost-7-tools-server-debuginfo
rhel-x86_64-ost-7-tools-server
rhel-x86_64-server-7-beta-debuginfo
rhel-x86_64-server-7-beta
rhel-x86_64-server-7-debuginfo
rhel-x86_64-server-extras-7-debuginfo
rhel-x86_64-server-extras-7
rhel-x86_64-server-ha-7-beta-debuginfo
rhel-x86_64-server-ha-7-beta
rhel-x86_64-server-ha-7-debuginfo
rhel-x86_64-server-ha-7
rhel-x86_64-server-optional-7-beta-debuginfo
rhel-x86_64-server-optional-7-beta
rhel-x86_64-server-optional-7-debuginfo
rhel-x86_64-server-optional-7
rhel-x86_64-server-rs-7-beta-debuginfo
rhel-x86_64-server-rs-7-beta
rhel-x86_64-server-rs-7-debuginfo
rhel-x86_64-server-rs-7
rhel-x86_64-server-supplementary-7-beta-debuginfo
rhel-x86_64-server-supplementary-7-beta
rhel-x86_64-server-supplementary-7-debuginfo
rhel-x86_64-server-supplementary-7
rhel-x86_64-server-v2vwin-7-debuginfo
rhel-x86_64-server-v2vwin-7
rhn-tools-rhel-x86_64-server-7-debuginfo
rhn-tools-rhel-x86_64-server-7

Product certificates installed successfully to /etc/pki/product.

Preparing to unregister system from legacy server...
System successfully unregistered from legacy server.
Stopping and disabling legacy services...
rhnsd.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig rhnsd off

Attempting to register system to destination server...
Registering to: subscription.rhsm.stage.redhat.com:443/subscription
The system has been registered with ID: f5fd7e04-1398-4986-a7c4-3cbacd849071
The registered system name is: kvm-06-guest02.hv2.lab.eng.bos.redhat.com
Installed Product Current Status:
Product Name: Red Hat Enterprise Linux for SAP
Status:       Subscribed

Product Name: Red Hat Enterprise Virtualization
Status:       Not Subscribed

Product Name: Red Hat OpenStack
Status:       Not Subscribed

Product Name: Red Hat Software Collections (for RHEL Server)
Status:       Subscribed

Product Name: Red Hat Software Collections Beta (for RHEL Server)
Status:       Subscribed

Product Name: Red Hat OpenStack Certification Test Suite
Status:       Not Subscribed

Product Name: Red Hat Enterprise MRG Messaging 2
Status:       Subscribed

Product Name: Red Hat Enterprise Linux for SAP Hana
Status:       Not Subscribed

Product Name: Red Hat Enterprise Linux for Real Time
Status:       Not Subscribed

Product Name: Red Hat Enterprise Linux for Real Time for NFV
Status:       Not Subscribed

Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

Product Name: Red Hat Enterprise Linux High Availability (for RHEL Server)
Status:       Subscribed

Product Name: Red Hat Enterprise Linux Resilient Storage (for RHEL Server)
Status:       Subscribed

Unable to find available subscriptions for all your installed products.
System 'kvm-06-guest02.hv2.lab.eng.bos.redhat.com' successfully registered.

[root@kvm-06-guest02 ~]# 


SUCCESS!  We were successfully migrated! Moreover, all of the conflicting product certificate mappings [83,208,69,191,90] resolved by picking one of the certificates and the tool installed them to /etc/pki/product/ as shown below...


[root@kvm-06-guest02 ~]# ls /etc/pki/product*
/etc/pki/product:
146.pem  150.pem  191.pem  201.pem  205.pem  208.pem  239.pem  241.pem  287.pem  313.pem  69.pem  83.pem  90.pem

/etc/pki/product-default:
69.pem
[root@kvm-06-guest02 ~]# 


VERIFIED: Moving this bug to VERIFIED as "conflicting product certificates" no longer impedes a successful migration from rhn classic to rhsm.

Comment 25 errata-xmlrpc 2020-09-29 19:22:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (subscription-manager bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3866