Bug 2026316

Summary: refresh command removes SCA certificate cache before it can be used
Product: Red Hat Enterprise Linux 9 Reporter: Rehana <redakkan>
Component: subscription-managerAssignee: Jiri Hnidek <jhnidek>
Status: CLOSED ERRATA QA Contact: Red Hat subscription-manager QE Team <rhsm-qe>
Severity: medium Docs Contact:
Priority: medium    
Version: 9.0CC: arpandey, candlepin-bugs, cdonnell, jhnidek, nmoumoul, redakkan, rhsm-qe
Target Milestone: rcKeywords: Triaged
Target Release: 9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: subscription-manager-1.29.23-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2024573 Environment:
Last Closed: 2022-05-17 15:58:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2024573, 2024589    
Bug Blocks:    

Description Rehana 2021-11-24 11:05:17 UTC 
+++ This bug was initially created as a clone of Bug #2024573 +++

Description of problem:
While being registered against an org with SCA enabled, running various subscription-manager commands, but most importantly 'refresh' and 'repos', subscription-manager deletes the SCA certificate cache, before it can be re-used (by sending the If-Modified-Since header to Candlepin).

Version-Release number of selected component (if applicable):

These are the ones I tried, but it's probably most versions of subman:
1.28.13-2.el8
1.29.18-1.fc33

How reproducible:
100%

Steps to Reproduce:
1. Register to an org with SCA mode
2. Validate that the /var/lib/rhsm/cache/content_access.json file is there, and has content.
3. Attach a debugger to the Candlepin server
4. Set debug points to these this endpoint: /consumers/{consumer_uuid}/accessible_content (The reason is to be able to stop execution between subman HTTP requests and check the client's filesystem for the cache file)
5. Run 'subscription-manager refresh' (or 'repos'
6. When the debugger stops on the '/accessible_content' call, check the /var/lib/rhsm/cache/content_access.json file on the client filesystem.

Actual results:
[nikos@localhost ~]$ sudo ls /var/lib/rhsm/cache/content_access.json
ls: cannot access '/var/lib/rhsm/cache/content_access.json': No such file or directory


Expected results:
The /var/lib/rhsm/cache/content_access.json file to be there, and populated properly.

Additional info:

If you continue the debugging session, you will see that once that call is done, only then is the cache re-created, and then only on the second call to /accessible_content the If-Modified-Since header is set.

If you want you can also set the two HTTP debug variables when running refresh: SUBMAN_DEBUG_PRINT_REQUEST=1 SUBMAN_DEBUG_PRINT_REQUEST_HEADER=1 and you will see that the first call to /accessible_content does not send the If-Modified-Since header (since the cache is simply not there):

Making insecure request: 192.168.122.179:8443 GET /candlepin/consumers/ab292bd8-3976-4fee-96b9-67ead1e13a13/accessible_content {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.28.13-2.el8', 'X-Correlation-ID': 'a10924cebda844859a83d5fbd20ce41a', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.28.13-2.el8', 'Content-Length': '0'}
...
Making insecure request: 192.168.122.179:8443 GET /candlepin/consumers/ab292bd8-3976-4fee-96b9-67ead1e13a13/accessible_content {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.28.13-2.el8', 'X-Correlation-ID': 'a10924cebda844859a83d5fbd20ce41a', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.28.13-2.el8', 'Content-Length': '0', 'If-Modified-Since': 'Thu, 18 Nov 2021 10:12:49 GMT'}
Comment 1 Archana Pandey 2022-01-21 09:09:59 UTC 
Pre-verification:

We will verify the issue by checking call to /accessible_content as the 'If-Modified-Since' header is not sent in the call to server if  cache is not there.

Reproducer:

[root@kvm-03-guest22 ~]# subscription-manager version
server type: This system is currently not registered.
subscription management server: 3.2.22-1
subscription management rules: 5.41
subscription-manager: 1.29.21-1.el9
[root@kvm-03-guest22 ~]# 
[root@kvm-03-guest22 ~]# subscription-manager register --username rhel9GA --password ***********
Registering to: subscription.rhsm.stage.redhat.com:443/subscription
The system has been registered with ID: e777edde-8415-497f-bff9-09cebfd5578d
The registered system name is: kvm-03-guest22.lab.eng.rdu2.redhat.com
[root@kvm-03-guest22 ~]# 
[root@kvm-03-guest22 ~]# cut -d ',' -f1 /var/lib/rhsm/cache/content_access.json
{"lastUpdate": "2022-01-21T08:57:56+0000"
[root@kvm-03-guest22 ~]# 
[root@kvm-03-guest22 ~]# export SUBMAN_DEBUG_PRINT_REQUEST=1 SUBMAN_DEBUG_PRINT_REQUEST_HEADER=1
[root@kvm-03-guest22 ~]# 
[root@kvm-03-guest22 ~]# subscription-manager refresh

Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/e777edde-8415-497f-bff9-09cebfd5578d/certificates/serials {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.21-1.el9', 'X-Correlation-ID': 'cac1f6e774b846389e11cff72af46892', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.21-1.el9', 'Content-Length': '0'}


Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/status {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.21-1.el9', 'X-Correlation-ID': 'cac1f6e774b846389e11cff72af46892', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.21-1.el9', 'Content-Length': '0'}


Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/e777edde-8415-497f-bff9-09cebfd5578d/accessible_content {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.21-1.el9', 'X-Correlation-ID': 'cac1f6e774b846389e11cff72af46892', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.21-1.el9', 'Content-Length': '0'}               <<<<<<<<<<<<< 'If-Modified-Since' header is not sent due to cache not being there


Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/e777edde-8415-497f-bff9-09cebfd5578d/content_overrides {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.21-1.el9', 'X-Correlation-ID': 'cac1f6e774b846389e11cff72af46892', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.21-1.el9', 'Content-Length': '0'}

All local data refreshed

Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/e777edde-8415-497f-bff9-09cebfd5578d/compliance {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.21-1.el9', 'X-Correlation-ID': 'cac1f6e774b846389e11cff72af46892', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.21-1.el9', 'Content-Length': '0'}

[root@kvm-03-guest22 ~]# 



===============================================================================
Pre-verifying on latest jenkins build

[root@kvm-03-guest22 ~]# yum upgrade subscription-manager
<output omitted >

oot@kvm-03-guest22 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 3.2.22-1
subscription management rules: 5.41
subscription-manager: 1.29.23-1.git.8.0557699.el9
[root@kvm-03-guest22 ~]# 
[root@kvm-03-guest22 ~]# export SUBMAN_DEBUG_PRINT_REQUEST=1 SUBMAN_DEBUG_PRINT_REQUEST_HEADER=1
[root@kvm-03-guest22 ~]# 
[root@kvm-03-guest22 ~]# subscription-manager refresh

Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/e777edde-8415-497f-bff9-09cebfd5578d/certificates/serials {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.23-1.git.8.0557699.el9', 'X-Correlation-ID': 'a2e7bb2e9a524710a83547b59d078106', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.23-1.git.8.0557699.el9', 'Content-Length': '0'}


Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/status {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.23-1.git.8.0557699.el9', 'X-Correlation-ID': 'a2e7bb2e9a524710a83547b59d078106', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.23-1.git.8.0557699.el9', 'Content-Length': '0'}


Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/e777edde-8415-497f-bff9-09cebfd5578d/accessible_content {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.23-1.git.8.0557699.el9', 'X-Correlation-ID': 'a2e7bb2e9a524710a83547b59d078106', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.23-1.git.8.0557699.el9', 'Content-Length': '0', 'If-Modified-Since': 'Fri, 21 Jan 2022 08:57:56 GMT'}
                                                                                                                     ^^^^^^^^^^ 'If-Modified-Since' header present
All local data refreshed
:
:
:
[root@kvm-03-guest22 ~]# 

Verification Results: 'If-Modified-Since' header is being sent in the request verifies that content_access.json cache is not removed in between.

Based on above observation setting verified field to tested.
Comment 4 Archana Pandey 2022-02-01 07:49:11 UTC 
verifying on subscription-manager-1.29.23-1.el9.x86_64

verifying the issue by checking presence of 'If-Modified-Since' header in '/accessible_content' call.

steps for verification : 

[root@dell-per630-fc-01 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 3.2.22-1
subscription management rules: 5.41
subscription-manager: 1.29.23-1.el9
[root@dell-per630-fc-01 ~]#

[root@dell-per630-fc-01 ~]# subscription-manager register --username rhel9GA --password redhat@123 
Registering to: subscription.rhsm.stage.redhat.com:443/subscription
The system has been registered with ID: 63f45c5b-9acd-45cb-8e2d-00210199cd50
The registered system name is: dell-per630-fc-01.dell2.lab.eng.bos.redhat.com

[root@dell-per630-fc-01 ~]# rpm -qa --changelog subscription-manager | grep 2026316
- 2026316: Do not delete cache of content_access during refresh
- 2026316: Do not delete cache of content_access during refresh

[root@dell-per630-fc-01 ~]# cut -d ',' -f1 /var/lib/rhsm/cache/content_access.json
{"lastUpdate": "2022-02-01T07:31:29+0000"
[root@dell-per630-fc-01 ~]# 
[root@dell-per630-fc-01 ~]# export SUBMAN_DEBUG_PRINT_REQUEST=1 SUBMAN_DEBUG_PRINT_REQUEST_HEADER=1
[root@dell-per630-fc-01 ~]# 
[root@dell-per630-fc-01 ~]# subscription-manager refresh

Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/63f45c5b-9acd-45cb-8e2d-00210199cd50/certificates/serials {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.23-1.el9', 'X-Correlation-ID': '79799a52e99e48be941800c648785e97', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.23-1.el9', 'Content-Length': '0'}


Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/status {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.23-1.el9', 'X-Correlation-ID': '79799a52e99e48be941800c648785e97', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.23-1.el9', 'Content-Length': '0'}


Making request: subscription.rhsm.stage.redhat.com:443 GET /subscription/consumers/63f45c5b-9acd-45cb-8e2d-00210199cd50/accessible_content {'Content-type': 'application/json', 'Accept': 'application/json', 'x-subscription-manager-version': '1.29.23-1.el9', 'X-Correlation-ID': '79799a52e99e48be941800c648785e97', 'Accept-Language': 'en-us', 'User-Agent': 'RHSM/1.0 (cmd=subscription-manager) subscription-manager/1.29.23-1.el9', 'Content-Length': '0', 'If-Modified-Since': 'Tue, 01 Feb 2022 07:31:29 GMT'}
                                                                                                ^^^^^-------------------------------------------header sent in request
All local data refreshed
:
:
:
[root@dell-per630-fc-01 ~]#

Verification Results: 'If-Modified-Since' header is being sent in the request verifies that content_access.json cache is not removed in between.
Comment 6 errata-xmlrpc 2022-05-17 15:58:19 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: subscription-manager), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:3984