Bug 1367243

Summary: 'Resource not found on the server' when running 'subscription-manager refresh'
Product: Red Hat Enterprise Linux 7 Reporter: qianzhan
Component: subscription-managerAssignee: Kevin Howell <khowell>
Status: CLOSED ERRATA QA Contact: John Sefler <jsefler>
Severity: high Docs Contact:
Priority: urgent    
Version: 7.3CC: bkearney, csnyder, hsun, khowell, qianzhan, redakkan, sgao, shihliu, skallesh, sreber, vrjain
Target Milestone: rcKeywords: Regression, Reopened
Target Release: 7.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 20:30:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
'rhsm.log'
none
resource_not_found_rhsm.log
none
satellite_rhsm.log none

Description qianzhan 2016-08-16 03:01:13 UTC
Description of problem:
'Resource not found on the server' when running 'subscription-manager refresh'

Version-Release number of selected component (if applicable):
subscription-manager: 1.17.10-1.el7
python-rhsm: 1.17.6-1.el7


How reproducible:
always

Steps to Reproduce:
1. Register and auto-attach RHEL-7.3-20160811.0 against SAM.

[root@dhcp-129-162 ~]# subscription-manager register --auto-attach
Registering to: samserv.redhat.com:443/sam/api
Username: admin
Password: 
The system has been registered with ID: d8f17fe3-ac2a-4dd5-aa25-f1995ae39d88 

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@dhcp-129-162 ~]# subscription-manager list --consumed
+-------------------------------------------+
   Consumed Subscriptions
+-------------------------------------------+
Subscription Name:   CloudForms (10-pack)
Provides:            Oracle Java (for RHEL Server)
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat CloudForms Beta
                     Red Hat Enterprise Linux Server
                     Red Hat CloudForms
                     Red Hat Beta
                     Red Hat Software Collections (for RHEL Server)
SKU:                 MCT2358
Contract:            10763966
Account:             5617732
Serial:              1733969557163054217
Pool ID:             8ac201185620973501562098c11302cf
Provides Management: Yes
Active:              True
Quantity Used:       1
Service Level:       Premium
Service Type:        L1-L3
Status Details:      Subscription is current
Subscription Type:   Standard
Starts:              05/30/2016
Ends:                05/30/2017
System Type:         Physical


2. Remove the entitlement certs.

[root@dhcp-129-162 ~]# rm -f /etc/pki/entitlement/*

3. Run refresh command.

[root@dhcp-129-162 ~]# subscription-manager refresh
Resource not found on the server
[root@dhcp-129-162 ~]# echo $?
70


Actual results:
As step 3, refreshing failed.

Expected results:
The refreshing should be successful with output 'All local data refreshed'

Additional info:
Please see attachment 'rhsm.log'

Comment 1 qianzhan 2016-08-16 03:02:36 UTC
Created attachment 1191043 [details]
'rhsm.log'

Comment 5 Chris Snyder 2016-08-31 17:35:48 UTC

*** This bug has been marked as a duplicate of bug 1366301 ***

Comment 6 qianzhan 2016-09-01 02:34:29 UTC
Move this bug as ASSIGNED:

1. Test subscription-manager-1.17.11-1.el7.x86_64 against SAM-1.4.1-RHEL-6-20141113.0:

[root@dhcp-129-48 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 1.4.3.28-1.el6sam_splice-Headpin
subscription management rules: Unknown
subscription-manager: 1.17.11-1.el7
python-rhsm: 1.17.6-1.el7


2. 
[root@dhcp-129-48 ~]# subscription-manager register --auto-attach
Registering to: samserv.redhat.com:443/sam/api
Username: admin
Password: 
The system has been registered with ID: 7a1a83fd-0a7e-4683-a624-d35b55905751 

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@dhcp-129-48 ~]# subscription-manager list --consumed
+-------------------------------------------+
   Consumed Subscriptions
+-------------------------------------------+
Subscription Name:   CloudForms (10-pack)
Provides:            Oracle Java (for RHEL Server)
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat CloudForms Beta
                     Red Hat Enterprise Linux Server
                     Red Hat CloudForms
                     Red Hat Beta
                     Red Hat Software Collections (for RHEL Server)
SKU:                 MCT2358
Contract:            10763966
Account:             5617732
Serial:              4840233098994396319
Pool ID:             8ac201185620973501562098c11302cf
Provides Management: Yes
Active:              True
Quantity Used:       1
Service Level:       Premium
Service Type:        L1-L3
Status Details:      Subscription is current
Subscription Type:   Standard
Starts:              05/30/2016
Ends:                05/29/2017
System Type:         Physical


[root@dhcp-129-48 ~]# rm -f /etc/pki/entitlement/*
[root@dhcp-129-48 ~]# subscription-manager refresh
Resource not found on the server
[root@dhcp-129-48 ~]# ll /etc/pki/entitlement/
total 0
[root@dhcp-129-48 ~]# subscription-manager list --consumed
+-------------------------------------------+
   Consumed Subscriptions
+-------------------------------------------+
Subscription Name:   CloudForms (10-pack)
Provides:            Oracle Java (for RHEL Server)
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat CloudForms Beta
                     Red Hat Enterprise Linux Server
                     Red Hat CloudForms
                     Red Hat Beta
                     Red Hat Software Collections (for RHEL Server)
SKU:                 MCT2358
Contract:            10763966
Account:             5617732
Serial:              4840233098994396319
Pool ID:             8ac201185620973501562098c11302cf
Provides Management: Yes
Active:              True
Quantity Used:       1
Service Level:       Premium
Service Type:        L1-L3
Status Details:      Subscription is current
Subscription Type:   Standard
Starts:              05/30/2016
Ends:                05/29/2017
System Type:         Physical

[root@dhcp-129-48 ~]# ll /etc/pki/entitlement/
total 20
-rw-------. 1 root root  1675 Aug 31 18:56 4840233098994396319-key.pem
-rw-r--r--. 1 root root 14665 Aug 31 18:56 4840233098994396319.pem

[root@dhcp-129-48 ~]# subscription-manager refresh
Resource not found on the server

Comment 7 qianzhan 2016-09-01 02:34:50 UTC
Created attachment 1196578 [details]
resource_not_found_rhsm.log

Comment 8 qianzhan 2016-09-01 02:37:00 UTC
Cancel this bug as a duplicate of bug 1366301, as bug 1366301 is verified.
Please see attachment 'resource_not_found_rhsm.log'

Comment 9 qianzhan 2016-09-01 08:37:14 UTC
Test subscription-manager-1.17.11-1.el7.x86_64 against Satellite-6.2.0-RHEL-7-20160831.0: Still failed with return code 70. Please see attachment 'satellite_rhsm.log'.



[root@hp-z220-05 ~]# subscription-manager register --auto-attach
Registering to: ent-02-vm-05.lab.eng.nay.redhat.com:443/rhsm
Username: admin
Password: 
The system has been registered with ID: 3c32a10c-0364-41ae-ba6e-66ebf60ebe17 

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@hp-z220-05 ~]# subscription-manager list --consumed
+-------------------------------------------+
   Consumed Subscriptions
+-------------------------------------------+
Subscription Name:   CloudForms (10-pack)
Provides:            Oracle Java (for RHEL Server)
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat CloudForms Beta
                     Red Hat Enterprise Linux Server
                     Red Hat CloudForms
                     Red Hat Beta
                     Red Hat Software Collections (for RHEL Server)
SKU:                 MCT2358
Contract:            10752428
Account:             5584289
Serial:              8856035383407880428
Pool ID:             8ac213af56e410410156e4228d570298
Provides Management: Yes
Active:              True
Quantity Used:       1
Service Level:       Premium
Service Type:        L1-L3
Status Details:      Subscription is current
Subscription Type:   Standard
Starts:              02/29/2016
Ends:                02/27/2017
System Type:         Physical

[root@hp-z220-05 ~]# ls /etc/pki/entitlement/
8856035383407880428-key.pem  8856035383407880428.pem
[root@hp-z220-05 ~]# rm -f /etc/pki/entitlement/*
[root@hp-z220-05 ~]# subscription-manager refresh
[root@hp-z220-05 ~]# echo $?
70
[root@hp-z220-05 ~]# subscription-manager list --consumed
No consumed subscription pools to list

Comment 10 qianzhan 2016-09-01 08:37:36 UTC
Created attachment 1196622 [details]
satellite_rhsm.log

Comment 11 John Sefler 2016-09-06 18:08:52 UTC
Please re-test comment 6 and comment 9 using python-rhsm-1.17.7-1

The fix in comment 5 for bug 1366301 included a fix for both components:
[root@jsefler-rhel7 ~]# rpm -q subscription-manager --changelog | grep 1366301 -B1
* Fri Aug 26 2016 Vritant Jain <vrjain> 1.17.11-1
- 1366301: Entitlement regeneration failure no longer aborts refresh
[root@jsefler-rhel7 ~]# 
[root@jsefler-rhel7 ~]# rpm -q python-rhsm --changelog | grep 1366301 -B1
* Fri Aug 26 2016 Vritant Jain <vrjain> 1.17.7-1
- 1366301: Entitlement regeneration no longer propagates server errors
[root@jsefler-rhel7 ~]#

Comment 12 qianzhan 2016-09-07 02:25:12 UTC
(In reply to John Sefler from comment #11)
> Please re-test comment 6 and comment 9 using python-rhsm-1.17.7-1
> 
> The fix in comment 5 for bug 1366301 included a fix for both components:
> [root@jsefler-rhel7 ~]# rpm -q subscription-manager --changelog | grep
> 1366301 -B1
> * Fri Aug 26 2016 Vritant Jain <vrjain> 1.17.11-1
> - 1366301: Entitlement regeneration failure no longer aborts refresh
> [root@jsefler-rhel7 ~]# 
> [root@jsefler-rhel7 ~]# rpm -q python-rhsm --changelog | grep 1366301 -B1
> * Fri Aug 26 2016 Vritant Jain <vrjain> 1.17.7-1
> - 1366301: Entitlement regeneration no longer propagates server errors
> [root@jsefler-rhel7 ~]#

Hi, John

This bug still exists with the components both against satellite and SAM:

1. Components version:RHEL-7.3-20160901.1(RHEL-7.3-Snapshot-2.0)

[root@dhcp-128-33 ~]# subscription-manager version
server type: This system is currently not registered.
subscription management server: 3.0.0.76-Unknown
subscription management rules: 5.15.2
subscription-manager: 1.17.11-1.el7
python-rhsm: 1.17.7-1.el7

2. Test results against Satellite-6.2.0-RHEL-7-20160831.0:
[root@dhcp-128-33 ~]# subscription-manager register --auto-attach
Registering to: sun-x8420-01.rhts.eng.bos.redhat.com:443/rhsm
Username: admin
Password: 
The system has been registered with ID: 42869f60-8f4a-4578-bc0b-4ef2350f9aff 

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@dhcp-128-33 ~]# ls /etc/pki/entitlement/
147395656340620675-key.pem  147395656340620675.pem
[root@dhcp-128-33 ~]# rm -f /etc/pki/entitlement/*
[root@dhcp-128-33 ~]# subscription-manager refresh
[root@dhcp-128-33 ~]# echo $?
70

3. Test results SAM-1.4.1-RHEL-6-20141113.0:
[root@dhcp-128-33 ~]# subscription-manager register --auto-attach
Registering to: samserv.redhat.com:443/sam/api
Username: admin
Password: 
The system has been registered with ID: 2cebbf6d-7d4f-41fc-8e00-789055a4f3d5 

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@dhcp-128-33 ~]# ls /etc/pki/entitlement/
3141217584309560348-key.pem  3141217584309560348.pem
[root@dhcp-128-33 ~]# rm -f /etc/pki/entitlement/*
[root@dhcp-128-33 ~]# subscription-manager refresh
Resource not found on the server
[root@dhcp-128-33 ~]# echo $?
70

Comment 13 Eko 2016-09-07 06:38:24 UTC
reproduced in RHEL-7.3-20160901.1(Snap2)
 - subscription-manager-1.17.11-1.el7.x86_64
 - python-rhsm-1.17.7-1.el7.x86_64

# subscription-manager  refresh
Resource not found on the server

Comment 14 Kevin Howell 2016-09-07 14:12:21 UTC
FYI, added safer fix, updated github accordingly

Comment 16 John Sefler 2016-09-08 14:19:05 UTC
Please re-test comment 6 and comment 9 against your SAM server using python-rhsm-1.17.9-1

[root@jsefler-rhel7 ~]# rpm -q python-rhsm --changelog | grep 1367243
- 1367243: Handle RestlibException 404 in refresh (khowell)
- 1367243: Fix 404 check in regen entitlement funcs (khowell)

Comment 17 qianzhan 2016-09-09 05:52:03 UTC
Verifying version:
subscription-manager: 1.17.14-1.el7
python-rhsm: 1.17.9-1.el7

1. Against Satellite-6.2.0-RHEL-7-20160907.6:
[root@dhcp-128-34 ~]# subscription-manager register --auto-attach
Registering to: ibm-x3655-05.ovirt.rhts.eng.bos.redhat.com:443/rhsm
Username: admin
Password: 
The system has been registered with ID: a6d5151d-66e5-4159-a05f-fdea75933eb6 

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@dhcp-128-34 ~]# ls /etc/pki/entitlement/
2040773249551249089-key.pem  2040773249551249089.pem
[root@dhcp-128-34 ~]# rm -f /etc/pki/entitlement/*
[root@dhcp-128-34 ~]# subscription-manager refresh
All local data refreshed
[root@dhcp-128-34 ~]# echo $?
0

2. Against SAM-1.4.1-RHEL-6-20141113.0:
[root@dhcp-128-34 ~]# subscription-manager register --auto-attach
Registering to: samserv.redhat.com:443/sam/api
Username: admin
Password: 
The system has been registered with ID: fc3f5ed6-8ab0-4d77-9eb9-25f25d5299db 

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@dhcp-128-34 ~]# ls /etc/pki/entitlement/
319678138210837357-key.pem  319678138210837357.pem
[root@dhcp-128-34 ~]# rm -f /etc/pki/entitlement/*
[root@dhcp-128-34 ~]# subscription-manager refresh
All local data refreshed
[root@dhcp-128-34 ~]# echo $?
0


Verified! Moving the bug form ON_QA to VERIFIED.

Comment 19 errata-xmlrpc 2016-11-03 20:30:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2592.html