Bug 1576582

Summary: rhsm configuration full_fresh_on_yum=1 is no longer working
Product: Red Hat Enterprise Linux 7 Reporter: John Sefler <jsefler>
Component: subscription-managerAssignee: Jiri Hnidek <jhnidek>
Status: CLOSED ERRATA QA Contact: Red Hat subscription-manager QE Team <rhsm-qe>
Severity: high Docs Contact:
Priority: high    
Version: 7.6CC: csnyder, jhnidek, khowell, redakkan
Target Milestone: rcKeywords: Regression, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: subscription-manager-1.21.5-4 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-30 10:36:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1122832    

Description John Sefler 2018-05-09 20:28:16 UTC 
Description of problem:
As a result of the work done on RFE Bug 1439645 (subscription-manager commit f944da840d756baf904f92705b4ed0425c96f1b5), the original behavior of full_fresh_on_yum=1 has regressed.  See test steps below.

After registering, subscribing, creating a repo-override, deleting the local repo-override cache, and setting full_fresh_on_yum=1, the deleted repo-override cache fails to be re-fetched from candlepin by the subscription-manager yum plugin.

I suspect commit f944da840d756baf904f92705b4ed0425c96f1b5 for RFE Bug 1439645 was bad.  After reading https://bugzilla.redhat.com/show_bug.cgi?id=1439645#c0, it sounds like RFE Bug 1439645 is asking that full_fresh_on_yum=1 also update the content sets assuming a change in the consumer object since server-side changes on environment stored on the consumer object.  I don't think that's what commit f944da840d756baf904f92705b4ed0425c96f1b5 implemented.


Version-Release number of selected component (if applicable):
[root@jsefler-rhel7 ~]# rpm -q subscription-manager
subscription-manager-1.21.4-3.git.0.5c7144b.el7.x86_64


How reproducible:


Steps to Reproduce:
[root@jsefler-rhel7 ~]# subscription-manager register --username=stage_auto_testuser --serverurl=subscription.rhsm.stage.redhat.com:443/subscription --auto-attach
Registering to: subscription.rhsm.stage.redhat.com:443/subscription
Password: 
The system has been registered with ID: d9c5df41-5847-44f8-ae91-ceb981b65a19
The registered system name is: jsefler-rhel7.usersys.redhat.com
Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@jsefler-rhel7 ~]# subscription-manager config --rhsm.full_refresh_on_yum=0

[root@jsefler-rhel7 ~]# subscription-manager repos --list-enabled | grep "Repo ID:"
Repo ID:   rhel-7-server-rpms

[root@jsefler-rhel7 ~]# subscription-manager repo-override --repo=rhel-7-server-rpms --add=retries:123

[root@jsefler-rhel7 ~]# grep -A13 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/3627598401438754986.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/3627598401438754986-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1
retries = 123

^^^^^^^ NOTICE: THE "retries" REPO-OVERRIDE HAS BEEN CORRECTLY ADDED

NOW LET'S REMOVE THE LOCAL CACHE AND ISSUE A YUM TRANSACTION (EXPECTING THE OVERRIDE TO DISAPPEAR BECAUSE REFRESH ON YUM IS OFF)

[root@jsefler-rhel7 ~]# rm -f /var/lib/rhsm/cache/content_overrides.json
[root@jsefler-rhel7 ~]# yum repolist
Loaded plugins: langpacks, product-id, rhnplugin, search-disabled-repos, subscription-manager
This system is not registered with RHN Classic or Red Hat Satellite.
You can use rhn_register to register.
Red Hat Satellite or RHN Classic support will be disabled.
repo id                                 repo name                                      status
rhel-7-server-rpms/7Server/x86_64       Red Hat Enterprise Linux 7 Server (RPMs)       20,133
repolist: 20,133

[root@jsefler-rhel7 ~]# grep -A12 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/3627598401438754986.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/3627598401438754986-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1

^^^^^^^^ ASSERTED: THE REPO OVERRIDE "retries" IS ABSENT AS EXPECTED

NOW LET'S TURN ON full_refresh_on_yum AND ISSUE A YUM TRANSACTION (EXPECTING IT TO REFRESH THE OVERRIDE FROM THE SERVER)

[root@jsefler-rhel7 ~]# subscription-manager config --rhsm.full_refresh_on_yum=1
[root@jsefler-rhel7 ~]# yum repolist
Loaded plugins: langpacks, product-id, rhnplugin, search-disabled-repos, subscription-manager
This system is not registered with RHN Classic or Red Hat Satellite.
You can use rhn_register to register.
Red Hat Satellite or RHN Classic support will be disabled.
repo id                                 repo name                                      status
rhel-7-server-rpms/7Server/x86_64       Red Hat Enterprise Linux 7 Server (RPMs)       20,133
repolist: 20,133

[root@jsefler-rhel7 ~]# grep -A12 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/3627598401438754986.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/3627598401438754986-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1

^^^^^^^^  FAILED: EXPECTED A TRAILING ENTRY FOR "retries = 123" TO HAVE BEEN DOWNLOADED BY THE subscription-manager YUM PLUGIN BECAUSE WE CONFIGURED rhsm.full_refresh_on_yum=1.
Comment 4 Rehana 2018-06-25 09:43:00 UTC 
Reproducing the failure :
==========================

# subscription-manager version
server type: This system is currently not registered.
subscription management server: 2.2.3-1
subscription management rules: 5.26
subscription-manager: 1.21.5-2.el7

# subscription-manager register --auto-attach
Registering to: subscription.rhsm.stage.redhat.com:443/subscription
Username: stage_test_rhel74
Password: 
The system has been registered with ID: d79a51ca-550f-45bd-9adc-7dc1744b692f
The registered system name is: kvm-01-guest18.rhts.eng.tlv.redhat.com
Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

# subscription-manager config --rhsm.full_refresh_on_yum=0

# subscription-manager repos --list-enabled | grep "Repo ID:"
Repo ID:   rhel-7-server-rpms

# subscription-manager repo-override --repo=rhel-7-server-rpms --add=retries:123

# grep -A13 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/2646577919102254778.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/2646577919102254778-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1
retries = 123

# rm -f /var/lib/rhsm/cache/content_overrides.json

#  yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
rhel-7-server-rpms/7Server/x86_64                                                                  Red Hat Enterprise Linux 7 Server (RPMs)                                                                  20,503
repolist: 58,410

# grep -A13 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/2646577919102254778.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/2646577919102254778-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1

^^Observed that repo-override "retries" did not appear in the redhat.repo with the full_refresh_on_yum disabled.

# subscription-manager config --rhsm.full_refresh_on_yum=1

# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id                                                                                            repo name                                                                                                 status
rhel-7-server-rpms/7Server/x86_64                                                                  Red Hat Enterprise Linux 7 Server (RPMs)                                                                  20,503
repolist: 20,503

# grep -A13 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/2646577919102254778.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/2646577919102254778-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1

^^ Reproduced the failure that even with " full_refresh_on_yum" set to "1" , the override "retried=123" was not updated in the redhat.repo file.
Comment 5 Rehana 2018-06-25 10:09:19 UTC 
Verifying on:
=============

Updated subscription-manager packages to the latest (not shown)
# subscription-manager version
server type: This system is currently not registered.
subscription management server: 2.3.8-1
subscription management rules: 5.26
subscription-manager: 1.21.5-4.el7

# rpm -qa --changelog subscription-manager | grep 1576582
- 1576582: Make rhsm.full_fresh_on_yum=1 working again (ENT-534)

Register and attach subscription on the system

# subscription-manager register --auto-attach
Registering to: subscription.rhsm.stage.redhat.com:443/subscription
Username: stage_test_rhel74
Password: 
The system has been registered with ID: 874d47e6-32ec-4be1-a0bb-18722795d160
The registered system name is: kvm-01-guest18.rhts.eng.tlv.redhat.com
Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

Set full_refresh_on_yum to '0'
# subscription-manager config --rhsm.full_refresh_on_yum=0

# subscription-manager repos --list-enabled | grep "Repo ID:"
Repo ID:   rhel-7-server-rpms

Add override

# subscription-manager repo-override --repo=rhel-7-server-rpms --add=retries:123

# grep -A13 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/2211907760137713583.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/2211907760137713583-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1
retries = 123
^^ notice the newly added repo-override is added in to the redhat.repo file

Remove the content_overrides_json file from the system
# rm -f /var/lib/rhsm/cache/content_overrides.json

# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id                                                                                            repo name                                                                                                 status
rhel-7-server-rpms/7Server/x86_64                                                                  Red Hat Enterprise Linux 7 Server (RPMs)                                                                  20,503
rhel76-Server                                                                                      rhel76-Server                                                                                              5,126
repolist: 25,629

# grep -A13 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/2211907760137713583.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/2211907760137713583-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1
^^ Notice the new override did not appear.

Now set the full_refresh_on_yum to "1"
# subscription-manager config --rhsm.full_refresh_on_yum=1

# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id                                                                                            repo name                                                                                                 status
rhel-7-server-rpms/7Server/x86_64                                                                  Red Hat Enterprise Linux 7 Server (RPMs)                                                                  20,503
rhel76-Server                                                                                      rhel76-Server                                                                                              5,126
repolist: 25,629

#  grep -A13 "\[rhel-7-server-rpms\]" /etc/yum.repos.d/redhat.repo 
[rhel-7-server-rpms]
metadata_expire = 86400
sslclientcert = /etc/pki/entitlement/2211907760137713583.pem
baseurl = https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
ui_repoid_vars = releasever basearch
sslverify = 1
name = Red Hat Enterprise Linux 7 Server (RPMs)
sslclientkey = /etc/pki/entitlement/2211907760137713583-key.pem
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
enabled = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
gpgcheck = 1
retries = 123

^^ Observed that with full_refresh_on_yum set to "1" , during the next yum transaction override "retries = 123" have been downloaded by the subscription-manager yum plugin.
Comment 7 errata-xmlrpc 2018-10-30 10:36:12 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3153